2023-02-13 16:54:17,288 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-13 16:54:17,528 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4 2023-02-13 16:54:17,542 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 16:54:17,542 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134, deleteOnExit=true 2023-02-13 16:54:17,542 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 16:54:17,543 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/test.cache.data in system properties and HBase conf 2023-02-13 16:54:17,543 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 16:54:17,543 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/hadoop.log.dir in system properties and HBase conf 2023-02-13 16:54:17,544 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 16:54:17,544 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 16:54:17,544 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 16:54:17,649 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 16:54:18,006 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 16:54:18,010 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 16:54:18,010 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 16:54:18,010 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 16:54:18,011 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 16:54:18,011 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 16:54:18,011 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 16:54:18,012 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 16:54:18,012 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 16:54:18,012 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 16:54:18,013 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/nfs.dump.dir in system properties and HBase conf 2023-02-13 16:54:18,013 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/java.io.tmpdir in system properties and HBase conf 2023-02-13 16:54:18,014 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 16:54:18,014 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 16:54:18,014 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 16:54:18,529 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 16:54:18,533 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 16:54:18,825 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 16:54:19,010 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 16:54:19,034 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 16:54:19,078 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 16:54:19,115 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/java.io.tmpdir/Jetty_localhost_39597_hdfs____.ycaull/webapp 2023-02-13 16:54:19,299 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39597 2023-02-13 16:54:19,309 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 16:54:19,310 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 16:54:19,686 WARN [Listener at localhost/33763] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 16:54:19,790 WARN [Listener at localhost/33763] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 16:54:19,813 WARN [Listener at localhost/33763] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 16:54:19,821 INFO [Listener at localhost/33763] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 16:54:19,828 INFO [Listener at localhost/33763] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/java.io.tmpdir/Jetty_localhost_36153_datanode____.kath3h/webapp 2023-02-13 16:54:19,928 INFO [Listener at localhost/33763] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36153 2023-02-13 16:54:20,274 WARN [Listener at localhost/42447] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 16:54:20,286 WARN [Listener at localhost/42447] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 16:54:20,290 WARN [Listener at localhost/42447] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 16:54:20,292 INFO [Listener at localhost/42447] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 16:54:20,297 INFO [Listener at localhost/42447] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/java.io.tmpdir/Jetty_localhost_36059_datanode____.3bdcso/webapp 2023-02-13 16:54:20,398 INFO [Listener at localhost/42447] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36059 2023-02-13 16:54:20,408 WARN [Listener at localhost/37117] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 16:54:20,430 WARN [Listener at localhost/37117] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 16:54:20,435 WARN [Listener at localhost/37117] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 16:54:20,436 INFO [Listener at localhost/37117] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 16:54:20,441 INFO [Listener at localhost/37117] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/java.io.tmpdir/Jetty_localhost_38949_datanode____.jm2sg/webapp 2023-02-13 16:54:20,539 INFO [Listener at localhost/37117] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38949 2023-02-13 16:54:20,548 WARN [Listener at localhost/38547] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 16:54:20,747 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4e19655d240b71d7: Processing first storage report for DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f from datanode a3585e58-faa4-4a64-84b5-04c6ddf75562 2023-02-13 16:54:20,749 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4e19655d240b71d7: from storage DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f node DatanodeRegistration(127.0.0.1:33991, datanodeUuid=a3585e58-faa4-4a64-84b5-04c6ddf75562, infoPort=42219, infoSecurePort=0, ipcPort=38547, storageInfo=lv=-57;cid=testClusterID;nsid=2128219518;c=1676307258605), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-13 16:54:20,749 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb48a8a4ca654ce0e: Processing first storage report for DS-23d5d252-2a95-4802-aee2-bc76b395b3df from datanode 1fadd7f6-283b-4c52-81b3-6a5231ac6c2e 2023-02-13 16:54:20,749 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb48a8a4ca654ce0e: from storage DS-23d5d252-2a95-4802-aee2-bc76b395b3df node DatanodeRegistration(127.0.0.1:42233, datanodeUuid=1fadd7f6-283b-4c52-81b3-6a5231ac6c2e, infoPort=33201, infoSecurePort=0, ipcPort=42447, storageInfo=lv=-57;cid=testClusterID;nsid=2128219518;c=1676307258605), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 16:54:20,749 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa0e6c86181906905: Processing first storage report for DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8 from datanode 14571ecd-b0a5-4d58-a36d-d2e6e3cf032c 2023-02-13 16:54:20,750 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa0e6c86181906905: from storage DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8 node DatanodeRegistration(127.0.0.1:45753, datanodeUuid=14571ecd-b0a5-4d58-a36d-d2e6e3cf032c, infoPort=34619, infoSecurePort=0, ipcPort=37117, storageInfo=lv=-57;cid=testClusterID;nsid=2128219518;c=1676307258605), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 16:54:20,750 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4e19655d240b71d7: Processing first storage report for DS-c4854d8e-0c4e-459b-9767-3bd0a1f8f696 from datanode a3585e58-faa4-4a64-84b5-04c6ddf75562 2023-02-13 16:54:20,750 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4e19655d240b71d7: from storage DS-c4854d8e-0c4e-459b-9767-3bd0a1f8f696 node DatanodeRegistration(127.0.0.1:33991, datanodeUuid=a3585e58-faa4-4a64-84b5-04c6ddf75562, infoPort=42219, infoSecurePort=0, ipcPort=38547, storageInfo=lv=-57;cid=testClusterID;nsid=2128219518;c=1676307258605), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 16:54:20,750 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb48a8a4ca654ce0e: Processing first storage report for DS-1b1b5cda-ea63-49df-b74a-76cdb3ff74a6 from datanode 1fadd7f6-283b-4c52-81b3-6a5231ac6c2e 2023-02-13 16:54:20,750 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb48a8a4ca654ce0e: from storage DS-1b1b5cda-ea63-49df-b74a-76cdb3ff74a6 node DatanodeRegistration(127.0.0.1:42233, datanodeUuid=1fadd7f6-283b-4c52-81b3-6a5231ac6c2e, infoPort=33201, infoSecurePort=0, ipcPort=42447, storageInfo=lv=-57;cid=testClusterID;nsid=2128219518;c=1676307258605), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 16:54:20,750 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa0e6c86181906905: Processing first storage report for DS-1a6f154a-83ac-4492-b4c0-2d86b01ca4a7 from datanode 14571ecd-b0a5-4d58-a36d-d2e6e3cf032c 2023-02-13 16:54:20,750 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa0e6c86181906905: from storage DS-1a6f154a-83ac-4492-b4c0-2d86b01ca4a7 node DatanodeRegistration(127.0.0.1:45753, datanodeUuid=14571ecd-b0a5-4d58-a36d-d2e6e3cf032c, infoPort=34619, infoSecurePort=0, ipcPort=37117, storageInfo=lv=-57;cid=testClusterID;nsid=2128219518;c=1676307258605), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 16:54:20,928 DEBUG [Listener at localhost/38547] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4 2023-02-13 16:54:20,993 INFO [Listener at localhost/38547] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/zookeeper_0, clientPort=57428, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 16:54:21,007 INFO [Listener at localhost/38547] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57428 2023-02-13 16:54:21,018 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:21,021 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:21,677 INFO [Listener at localhost/38547] util.FSUtils(479): Created version file at hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c with version=8 2023-02-13 16:54:21,677 INFO [Listener at localhost/38547] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/hbase-staging 2023-02-13 16:54:22,002 INFO [Listener at localhost/38547] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 16:54:22,470 INFO [Listener at localhost/38547] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 16:54:22,507 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:22,508 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:22,509 INFO [Listener at localhost/38547] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 16:54:22,509 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:22,509 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 16:54:22,648 INFO [Listener at localhost/38547] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 16:54:22,718 DEBUG [Listener at localhost/38547] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 16:54:22,815 INFO [Listener at localhost/38547] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45273 2023-02-13 16:54:22,825 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:22,827 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:22,847 INFO [Listener at localhost/38547] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45273 connecting to ZooKeeper ensemble=127.0.0.1:57428 2023-02-13 16:54:22,886 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:452730x0, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 16:54:22,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:45273-0x1004e9acff70000 connected 2023-02-13 16:54:22,914 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 16:54:22,915 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:54:22,918 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 16:54:22,926 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45273 2023-02-13 16:54:22,927 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45273 2023-02-13 16:54:22,927 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45273 2023-02-13 16:54:22,928 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45273 2023-02-13 16:54:22,928 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45273 2023-02-13 16:54:22,934 INFO [Listener at localhost/38547] master.HMaster(439): hbase.rootdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c, hbase.cluster.distributed=false 2023-02-13 16:54:22,999 INFO [Listener at localhost/38547] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 16:54:22,999 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:22,999 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:22,999 INFO [Listener at localhost/38547] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 16:54:22,999 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:22,999 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 16:54:23,004 INFO [Listener at localhost/38547] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 16:54:23,007 INFO [Listener at localhost/38547] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41587 2023-02-13 16:54:23,009 INFO [Listener at localhost/38547] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 16:54:23,015 DEBUG [Listener at localhost/38547] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 16:54:23,016 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:23,018 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:23,020 INFO [Listener at localhost/38547] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41587 connecting to ZooKeeper ensemble=127.0.0.1:57428 2023-02-13 16:54:23,024 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:415870x0, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 16:54:23,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41587-0x1004e9acff70001 connected 2023-02-13 16:54:23,025 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 16:54:23,026 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:54:23,027 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 16:54:23,028 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41587 2023-02-13 16:54:23,028 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41587 2023-02-13 16:54:23,028 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41587 2023-02-13 16:54:23,029 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41587 2023-02-13 16:54:23,029 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41587 2023-02-13 16:54:23,042 INFO [Listener at localhost/38547] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 16:54:23,042 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:23,043 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:23,043 INFO [Listener at localhost/38547] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 16:54:23,043 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:23,044 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 16:54:23,044 INFO [Listener at localhost/38547] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 16:54:23,046 INFO [Listener at localhost/38547] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38051 2023-02-13 16:54:23,046 INFO [Listener at localhost/38547] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 16:54:23,047 DEBUG [Listener at localhost/38547] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 16:54:23,048 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:23,051 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:23,054 INFO [Listener at localhost/38547] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38051 connecting to ZooKeeper ensemble=127.0.0.1:57428 2023-02-13 16:54:23,060 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:380510x0, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 16:54:23,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:38051-0x1004e9acff70002 connected 2023-02-13 16:54:23,061 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 16:54:23,062 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:54:23,062 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 16:54:23,063 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38051 2023-02-13 16:54:23,063 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38051 2023-02-13 16:54:23,064 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38051 2023-02-13 16:54:23,064 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38051 2023-02-13 16:54:23,064 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38051 2023-02-13 16:54:23,077 INFO [Listener at localhost/38547] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 16:54:23,077 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:23,077 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:23,077 INFO [Listener at localhost/38547] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 16:54:23,077 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:54:23,077 INFO [Listener at localhost/38547] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 16:54:23,077 INFO [Listener at localhost/38547] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 16:54:23,079 INFO [Listener at localhost/38547] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34717 2023-02-13 16:54:23,079 INFO [Listener at localhost/38547] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 16:54:23,080 DEBUG [Listener at localhost/38547] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 16:54:23,081 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:23,083 INFO [Listener at localhost/38547] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:23,084 INFO [Listener at localhost/38547] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34717 connecting to ZooKeeper ensemble=127.0.0.1:57428 2023-02-13 16:54:23,087 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:347170x0, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 16:54:23,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:34717-0x1004e9acff70003 connected 2023-02-13 16:54:23,088 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 16:54:23,088 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:54:23,089 DEBUG [Listener at localhost/38547] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 16:54:23,090 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34717 2023-02-13 16:54:23,090 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34717 2023-02-13 16:54:23,090 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34717 2023-02-13 16:54:23,091 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34717 2023-02-13 16:54:23,091 DEBUG [Listener at localhost/38547] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34717 2023-02-13 16:54:23,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:54:23,101 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 16:54:23,102 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:54:23,119 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 16:54:23,119 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 16:54:23,119 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 16:54:23,119 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 16:54:23,120 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:54:23,121 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 16:54:23,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,45273,1676307261815 from backup master directory 2023-02-13 16:54:23,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 16:54:23,128 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:54:23,128 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 16:54:23,128 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 16:54:23,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:54:23,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 16:54:23,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 16:54:23,221 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/hbase.id with ID: f0f9445d-baa7-4d25-9b96-e828be704660 2023-02-13 16:54:23,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:54:23,289 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:54:23,340 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x25e220f6 to 127.0.0.1:57428 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:23,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1aa22bae, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:23,394 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 16:54:23,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 16:54:23,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 16:54:23,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 16:54:23,414 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:54:23,418 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:54:23,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 16:54:23,452 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/data/master/store-tmp 2023-02-13 16:54:23,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:23,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 16:54:23,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:54:23,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:54:23,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 16:54:23,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:54:23,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:54:23,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676307263515Disabling compacts and flushes for region at 1676307263516 (+1 ms)Disabling writes for close at 1676307263516Writing region close event to WAL at 1676307263516Closed at 1676307263516 2023-02-13 16:54:23,518 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/WALs/jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:54:23,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45273%2C1676307261815, suffix=, logDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/WALs/jenkins-hbase4.apache.org,45273,1676307261815, archiveDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/oldWALs, maxLogs=10 2023-02-13 16:54:23,591 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK] 2023-02-13 16:54:23,591 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK] 2023-02-13 16:54:23,591 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK] 2023-02-13 16:54:23,599 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:54:23,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/WALs/jenkins-hbase4.apache.org,45273,1676307261815/jenkins-hbase4.apache.org%2C45273%2C1676307261815.1676307263547 2023-02-13 16:54:23,684 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK], DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK], DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK]] 2023-02-13 16:54:23,684 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:23,685 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:23,688 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:54:23,689 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:54:23,745 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:54:23,752 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 16:54:23,775 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 16:54:23,786 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:23,791 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:54:23,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:54:23,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:54:23,812 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:23,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63182857, jitterRate=-0.058502063155174255}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:23,814 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676307263715Initializing all the Stores at 1676307263717 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307263718 (+1 ms)Cleaning up temporary data from old regions at 1676307263800 (+82 ms)Cleaning up detritus from prior splits at 1676307263801 (+1 ms)Region opened successfully at 1676307263814 (+13 ms) 2023-02-13 16:54:23,815 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 16:54:23,834 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:23,834 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:23,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:23,838 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 16:54:23,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-02-13 16:54:23,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 16:54:23,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 16:54:23,899 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 16:54:23,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 16:54:23,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 16:54:23,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 16:54:23,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 16:54:23,935 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 16:54:23,938 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:54:23,939 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 16:54:23,940 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 16:54:23,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 16:54:23,959 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 16:54:23,959 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 16:54:23,959 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 16:54:23,959 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 16:54:23,959 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:54:23,960 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,45273,1676307261815, sessionid=0x1004e9acff70000, setting cluster-up flag (Was=false) 2023-02-13 16:54:23,974 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:54:23,980 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 16:54:23,981 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:54:23,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:54:23,991 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 16:54:23,992 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:54:23,995 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.hbase-snapshot/.tmp 2023-02-13 16:54:24,095 INFO [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(952): ClusterId : f0f9445d-baa7-4d25-9b96-e828be704660 2023-02-13 16:54:24,095 INFO [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(952): ClusterId : f0f9445d-baa7-4d25-9b96-e828be704660 2023-02-13 16:54:24,096 INFO [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(952): ClusterId : f0f9445d-baa7-4d25-9b96-e828be704660 2023-02-13 16:54:24,102 DEBUG [RS:1;jenkins-hbase4:38051] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 16:54:24,102 DEBUG [RS:0;jenkins-hbase4:41587] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 16:54:24,102 DEBUG [RS:2;jenkins-hbase4:34717] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 16:54:24,109 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:24,109 DEBUG [RS:2;jenkins-hbase4:34717] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 16:54:24,109 DEBUG [RS:0;jenkins-hbase4:41587] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 16:54:24,109 DEBUG [RS:1;jenkins-hbase4:38051] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 16:54:24,109 DEBUG [RS:1;jenkins-hbase4:38051] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 16:54:24,109 DEBUG [RS:0;jenkins-hbase4:41587] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 16:54:24,109 DEBUG [RS:2;jenkins-hbase4:34717] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 16:54:24,113 DEBUG [RS:0;jenkins-hbase4:41587] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 16:54:24,113 DEBUG [RS:1;jenkins-hbase4:38051] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 16:54:24,113 DEBUG [RS:2;jenkins-hbase4:34717] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 16:54:24,115 DEBUG [RS:0;jenkins-hbase4:41587] zookeeper.ReadOnlyZKClient(139): Connect 0x6db8f005 to 127.0.0.1:57428 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:24,116 DEBUG [RS:2;jenkins-hbase4:34717] zookeeper.ReadOnlyZKClient(139): Connect 0x0e28de38 to 127.0.0.1:57428 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:24,116 DEBUG [RS:1;jenkins-hbase4:38051] zookeeper.ReadOnlyZKClient(139): Connect 0x5a6d36a2 to 127.0.0.1:57428 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:24,125 DEBUG [RS:2;jenkins-hbase4:34717] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@ef318d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:24,125 DEBUG [RS:0;jenkins-hbase4:41587] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@637f03c2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:24,125 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 16:54:24,125 DEBUG [RS:2;jenkins-hbase4:34717] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d0b548, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 16:54:24,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 16:54:24,125 DEBUG [RS:1;jenkins-hbase4:38051] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@54aac39f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:24,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 16:54:24,126 DEBUG [RS:0;jenkins-hbase4:41587] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@67409b75, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 16:54:24,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 16:54:24,126 DEBUG [RS:1;jenkins-hbase4:38051] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@50955dd2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 16:54:24,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-13 16:54:24,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 16:54:24,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307294129 2023-02-13 16:54:24,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 16:54:24,135 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:24,135 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 16:54:24,139 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 16:54:24,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 16:54:24,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 16:54:24,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 16:54:24,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 16:54:24,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 16:54:24,150 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:38051 2023-02-13 16:54:24,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,153 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:34717 2023-02-13 16:54:24,154 DEBUG [RS:0;jenkins-hbase4:41587] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:41587 2023-02-13 16:54:24,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 16:54:24,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 16:54:24,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 16:54:24,158 INFO [RS:0;jenkins-hbase4:41587] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 16:54:24,158 INFO [RS:2;jenkins-hbase4:34717] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 16:54:24,159 INFO [RS:0;jenkins-hbase4:41587] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 16:54:24,158 INFO [RS:1;jenkins-hbase4:38051] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 16:54:24,159 DEBUG [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 16:54:24,159 INFO [RS:2;jenkins-hbase4:34717] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 16:54:24,159 INFO [RS:1;jenkins-hbase4:38051] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 16:54:24,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 16:54:24,159 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 16:54:24,159 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 16:54:24,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 16:54:24,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676307264161,5,FailOnTimeoutGroup] 2023-02-13 16:54:24,162 INFO [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,45273,1676307261815 with isa=jenkins-hbase4.apache.org/172.31.14.131:41587, startcode=1676307262998 2023-02-13 16:54:24,162 INFO [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,45273,1676307261815 with isa=jenkins-hbase4.apache.org/172.31.14.131:38051, startcode=1676307263041 2023-02-13 16:54:24,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676307264162,5,FailOnTimeoutGroup] 2023-02-13 16:54:24,163 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,163 INFO [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,45273,1676307261815 with isa=jenkins-hbase4.apache.org/172.31.14.131:34717, startcode=1676307263076 2023-02-13 16:54:24,163 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 16:54:24,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,189 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:24,190 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:24,190 DEBUG [RS:2;jenkins-hbase4:34717] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 16:54:24,191 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c 2023-02-13 16:54:24,190 DEBUG [RS:0;jenkins-hbase4:41587] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 16:54:24,190 DEBUG [RS:1;jenkins-hbase4:38051] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 16:54:24,218 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:24,222 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 16:54:24,228 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/info 2023-02-13 16:54:24,229 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 16:54:24,230 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:24,230 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 16:54:24,235 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/rep_barrier 2023-02-13 16:54:24,235 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 16:54:24,236 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:24,236 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 16:54:24,239 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/table 2023-02-13 16:54:24,239 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 16:54:24,240 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:24,243 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740 2023-02-13 16:54:24,244 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740 2023-02-13 16:54:24,248 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 16:54:24,250 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 16:54:24,257 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:24,257 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69470370, jitterRate=0.0351891815662384}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 16:54:24,257 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676307264219Initializing all the Stores at 1676307264221 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307264221Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307264222 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307264222Cleaning up temporary data from old regions at 1676307264246 (+24 ms)Cleaning up detritus from prior splits at 1676307264246Region opened successfully at 1676307264257 (+11 ms) 2023-02-13 16:54:24,258 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 16:54:24,258 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 16:54:24,258 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 16:54:24,258 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 16:54:24,258 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 16:54:24,260 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 16:54:24,261 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676307264258Disabling compacts and flushes for region at 1676307264258Disabling writes for close at 1676307264258Writing region close event to WAL at 1676307264260 (+2 ms)Closed at 1676307264260 2023-02-13 16:54:24,261 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41735, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 16:54:24,261 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57931, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 16:54:24,261 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54821, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 16:54:24,267 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:24,267 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 16:54:24,275 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:24,275 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45273] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:24,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45273] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:24,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 16:54:24,295 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c 2023-02-13 16:54:24,295 DEBUG [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c 2023-02-13 16:54:24,295 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c 2023-02-13 16:54:24,296 DEBUG [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:33763 2023-02-13 16:54:24,296 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:33763 2023-02-13 16:54:24,296 DEBUG [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 16:54:24,296 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:33763 2023-02-13 16:54:24,296 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 16:54:24,296 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 16:54:24,297 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 16:54:24,299 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 16:54:24,306 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 16:54:24,306 DEBUG [RS:2;jenkins-hbase4:34717] zookeeper.ZKUtil(162): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:24,307 DEBUG [RS:1;jenkins-hbase4:38051] zookeeper.ZKUtil(162): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:24,307 WARN [RS:2;jenkins-hbase4:34717] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 16:54:24,307 WARN [RS:1;jenkins-hbase4:38051] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 16:54:24,307 DEBUG [RS:0;jenkins-hbase4:41587] zookeeper.ZKUtil(162): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:24,307 INFO [RS:1;jenkins-hbase4:38051] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 16:54:24,308 WARN [RS:0;jenkins-hbase4:41587] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 16:54:24,307 INFO [RS:2;jenkins-hbase4:34717] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 16:54:24,308 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1947): logDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:24,308 INFO [RS:0;jenkins-hbase4:41587] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 16:54:24,308 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38051,1676307263041] 2023-02-13 16:54:24,308 DEBUG [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(1947): logDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:24,308 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1947): logDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:24,308 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34717,1676307263076] 2023-02-13 16:54:24,308 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41587,1676307262998] 2023-02-13 16:54:24,321 DEBUG [RS:1;jenkins-hbase4:38051] zookeeper.ZKUtil(162): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:24,321 DEBUG [RS:0;jenkins-hbase4:41587] zookeeper.ZKUtil(162): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:24,321 DEBUG [RS:2;jenkins-hbase4:34717] zookeeper.ZKUtil(162): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:24,322 DEBUG [RS:1;jenkins-hbase4:38051] zookeeper.ZKUtil(162): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:24,322 DEBUG [RS:0;jenkins-hbase4:41587] zookeeper.ZKUtil(162): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:24,322 DEBUG [RS:2;jenkins-hbase4:34717] zookeeper.ZKUtil(162): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:24,322 DEBUG [RS:1;jenkins-hbase4:38051] zookeeper.ZKUtil(162): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:24,322 DEBUG [RS:0;jenkins-hbase4:41587] zookeeper.ZKUtil(162): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:24,322 DEBUG [RS:2;jenkins-hbase4:34717] zookeeper.ZKUtil(162): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:24,333 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 16:54:24,333 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 16:54:24,333 DEBUG [RS:0;jenkins-hbase4:41587] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 16:54:24,342 INFO [RS:0;jenkins-hbase4:41587] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 16:54:24,342 INFO [RS:1;jenkins-hbase4:38051] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 16:54:24,342 INFO [RS:2;jenkins-hbase4:34717] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 16:54:24,362 INFO [RS:0;jenkins-hbase4:41587] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 16:54:24,362 INFO [RS:2;jenkins-hbase4:34717] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 16:54:24,362 INFO [RS:1;jenkins-hbase4:38051] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 16:54:24,366 INFO [RS:0;jenkins-hbase4:41587] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:54:24,366 INFO [RS:2;jenkins-hbase4:34717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:54:24,366 INFO [RS:0;jenkins-hbase4:41587] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,366 INFO [RS:1;jenkins-hbase4:38051] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:54:24,366 INFO [RS:2;jenkins-hbase4:34717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,367 INFO [RS:1;jenkins-hbase4:38051] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,367 INFO [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 16:54:24,367 INFO [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 16:54:24,367 INFO [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 16:54:24,375 INFO [RS:0;jenkins-hbase4:41587] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,375 INFO [RS:2;jenkins-hbase4:34717] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,375 INFO [RS:1;jenkins-hbase4:38051] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,375 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,375 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,376 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,376 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,376 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,376 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,376 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,376 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 16:54:24,377 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 16:54:24,377 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,376 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:0;jenkins-hbase4:41587] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,377 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,378 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,378 DEBUG [RS:1;jenkins-hbase4:38051] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,378 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,378 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 16:54:24,379 INFO [RS:0;jenkins-hbase4:41587] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,379 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,379 INFO [RS:0;jenkins-hbase4:41587] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,379 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,379 INFO [RS:0;jenkins-hbase4:41587] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,379 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,379 INFO [RS:1;jenkins-hbase4:38051] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,379 DEBUG [RS:2;jenkins-hbase4:34717] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:54:24,379 INFO [RS:1;jenkins-hbase4:38051] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,379 INFO [RS:1;jenkins-hbase4:38051] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,383 INFO [RS:2;jenkins-hbase4:34717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,383 INFO [RS:2;jenkins-hbase4:34717] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,383 INFO [RS:2;jenkins-hbase4:34717] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,395 INFO [RS:1;jenkins-hbase4:38051] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 16:54:24,395 INFO [RS:0;jenkins-hbase4:41587] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 16:54:24,399 INFO [RS:0;jenkins-hbase4:41587] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41587,1676307262998-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,399 INFO [RS:1;jenkins-hbase4:38051] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38051,1676307263041-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,399 INFO [RS:2;jenkins-hbase4:34717] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 16:54:24,399 INFO [RS:2;jenkins-hbase4:34717] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34717,1676307263076-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,418 INFO [RS:1;jenkins-hbase4:38051] regionserver.Replication(203): jenkins-hbase4.apache.org,38051,1676307263041 started 2023-02-13 16:54:24,418 INFO [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,38051,1676307263041, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38051, sessionid=0x1004e9acff70002 2023-02-13 16:54:24,418 DEBUG [RS:1;jenkins-hbase4:38051] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 16:54:24,418 DEBUG [RS:1;jenkins-hbase4:38051] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:24,418 DEBUG [RS:1;jenkins-hbase4:38051] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38051,1676307263041' 2023-02-13 16:54:24,419 DEBUG [RS:1;jenkins-hbase4:38051] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:24,419 INFO [RS:0;jenkins-hbase4:41587] regionserver.Replication(203): jenkins-hbase4.apache.org,41587,1676307262998 started 2023-02-13 16:54:24,419 INFO [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,41587,1676307262998, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41587, sessionid=0x1004e9acff70001 2023-02-13 16:54:24,419 INFO [RS:2;jenkins-hbase4:34717] regionserver.Replication(203): jenkins-hbase4.apache.org,34717,1676307263076 started 2023-02-13 16:54:24,419 DEBUG [RS:0;jenkins-hbase4:41587] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 16:54:24,419 INFO [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,34717,1676307263076, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34717, sessionid=0x1004e9acff70003 2023-02-13 16:54:24,419 DEBUG [RS:0;jenkins-hbase4:41587] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:24,420 DEBUG [RS:2;jenkins-hbase4:34717] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 16:54:24,420 DEBUG [RS:2;jenkins-hbase4:34717] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:24,420 DEBUG [RS:2;jenkins-hbase4:34717] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34717,1676307263076' 2023-02-13 16:54:24,420 DEBUG [RS:2;jenkins-hbase4:34717] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:24,420 DEBUG [RS:0;jenkins-hbase4:41587] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41587,1676307262998' 2023-02-13 16:54:24,420 DEBUG [RS:1;jenkins-hbase4:38051] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:24,421 DEBUG [RS:0;jenkins-hbase4:41587] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:24,421 DEBUG [RS:2;jenkins-hbase4:34717] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:24,421 DEBUG [RS:0;jenkins-hbase4:41587] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:24,421 DEBUG [RS:1;jenkins-hbase4:38051] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 16:54:24,421 DEBUG [RS:1;jenkins-hbase4:38051] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 16:54:24,421 DEBUG [RS:1;jenkins-hbase4:38051] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:24,421 DEBUG [RS:1;jenkins-hbase4:38051] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38051,1676307263041' 2023-02-13 16:54:24,421 DEBUG [RS:1;jenkins-hbase4:38051] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 16:54:24,421 DEBUG [RS:2;jenkins-hbase4:34717] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 16:54:24,422 DEBUG [RS:0;jenkins-hbase4:41587] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 16:54:24,422 DEBUG [RS:0;jenkins-hbase4:41587] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 16:54:24,422 DEBUG [RS:2;jenkins-hbase4:34717] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 16:54:24,422 DEBUG [RS:2;jenkins-hbase4:34717] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:24,422 DEBUG [RS:0;jenkins-hbase4:41587] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:24,422 DEBUG [RS:0;jenkins-hbase4:41587] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41587,1676307262998' 2023-02-13 16:54:24,422 DEBUG [RS:0;jenkins-hbase4:41587] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 16:54:24,422 DEBUG [RS:2;jenkins-hbase4:34717] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34717,1676307263076' 2023-02-13 16:54:24,422 DEBUG [RS:1;jenkins-hbase4:38051] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 16:54:24,422 DEBUG [RS:2;jenkins-hbase4:34717] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 16:54:24,423 DEBUG [RS:0;jenkins-hbase4:41587] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 16:54:24,423 DEBUG [RS:1;jenkins-hbase4:38051] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 16:54:24,423 DEBUG [RS:2;jenkins-hbase4:34717] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 16:54:24,423 INFO [RS:1;jenkins-hbase4:38051] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 16:54:24,423 DEBUG [RS:0;jenkins-hbase4:41587] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 16:54:24,423 INFO [RS:1;jenkins-hbase4:38051] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 16:54:24,423 INFO [RS:0;jenkins-hbase4:41587] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 16:54:24,424 INFO [RS:0;jenkins-hbase4:41587] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 16:54:24,424 DEBUG [RS:2;jenkins-hbase4:34717] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 16:54:24,424 INFO [RS:2;jenkins-hbase4:34717] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 16:54:24,425 INFO [RS:2;jenkins-hbase4:34717] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 16:54:24,451 DEBUG [jenkins-hbase4:45273] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 16:54:24,454 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:54:24,460 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:54:24,460 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:54:24,460 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:54:24,460 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:54:24,463 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34717,1676307263076, state=OPENING 2023-02-13 16:54:24,470 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 16:54:24,471 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:54:24,472 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 16:54:24,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:24,535 INFO [RS:1;jenkins-hbase4:38051] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38051%2C1676307263041, suffix=, logDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,38051,1676307263041, archiveDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/oldWALs, maxLogs=32 2023-02-13 16:54:24,536 INFO [RS:0;jenkins-hbase4:41587] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41587%2C1676307262998, suffix=, logDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,41587,1676307262998, archiveDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/oldWALs, maxLogs=32 2023-02-13 16:54:24,535 INFO [RS:2;jenkins-hbase4:34717] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34717%2C1676307263076, suffix=, logDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,34717,1676307263076, archiveDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/oldWALs, maxLogs=32 2023-02-13 16:54:24,561 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK] 2023-02-13 16:54:24,561 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK] 2023-02-13 16:54:24,561 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK] 2023-02-13 16:54:24,562 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK] 2023-02-13 16:54:24,562 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK] 2023-02-13 16:54:24,564 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK] 2023-02-13 16:54:24,566 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK] 2023-02-13 16:54:24,567 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK] 2023-02-13 16:54:24,568 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK] 2023-02-13 16:54:24,576 INFO [RS:1;jenkins-hbase4:38051] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,38051,1676307263041/jenkins-hbase4.apache.org%2C38051%2C1676307263041.1676307264541 2023-02-13 16:54:24,577 INFO [RS:2;jenkins-hbase4:34717] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,34717,1676307263076/jenkins-hbase4.apache.org%2C34717%2C1676307263076.1676307264541 2023-02-13 16:54:24,577 INFO [RS:0;jenkins-hbase4:41587] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,41587,1676307262998/jenkins-hbase4.apache.org%2C41587%2C1676307262998.1676307264541 2023-02-13 16:54:24,579 DEBUG [RS:1;jenkins-hbase4:38051] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK], DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK], DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK]] 2023-02-13 16:54:24,579 DEBUG [RS:2;jenkins-hbase4:34717] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK], DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK], DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK]] 2023-02-13 16:54:24,579 DEBUG [RS:0;jenkins-hbase4:41587] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK], DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK], DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK]] 2023-02-13 16:54:24,658 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:24,661 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 16:54:24,663 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48862, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 16:54:24,679 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 16:54:24,680 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 16:54:24,683 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34717%2C1676307263076.meta, suffix=.meta, logDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,34717,1676307263076, archiveDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/oldWALs, maxLogs=32 2023-02-13 16:54:24,704 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK] 2023-02-13 16:54:24,704 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK] 2023-02-13 16:54:24,706 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK] 2023-02-13 16:54:24,713 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/WALs/jenkins-hbase4.apache.org,34717,1676307263076/jenkins-hbase4.apache.org%2C34717%2C1676307263076.meta.1676307264685.meta 2023-02-13 16:54:24,713 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45753,DS-a7a24a6a-3095-4e82-983c-c1be5b14b3f8,DISK], DatanodeInfoWithStorage[127.0.0.1:33991,DS-fca6a18c-e07d-4422-82fe-65c1d1cbba7f,DISK], DatanodeInfoWithStorage[127.0.0.1:42233,DS-23d5d252-2a95-4802-aee2-bc76b395b3df,DISK]] 2023-02-13 16:54:24,713 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:24,718 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:24,718 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 16:54:24,736 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 16:54:24,738 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 16:54:24,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 16:54:24,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:24,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 16:54:24,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 16:54:24,746 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 16:54:24,748 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/info 2023-02-13 16:54:24,748 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/info 2023-02-13 16:54:24,749 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 16:54:24,749 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:24,750 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 16:54:24,751 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/rep_barrier 2023-02-13 16:54:24,751 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/rep_barrier 2023-02-13 16:54:24,752 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 16:54:24,753 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:24,753 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 16:54:24,754 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/table 2023-02-13 16:54:24,754 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/table 2023-02-13 16:54:24,755 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 16:54:24,755 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:24,757 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740 2023-02-13 16:54:24,760 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740 2023-02-13 16:54:24,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 16:54:24,766 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 16:54:24,767 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60612557, jitterRate=-0.09680251777172089}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 16:54:24,767 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676307264743Writing region info on filesystem at 1676307264744 (+1 ms)Initializing all the Stores at 1676307264745 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307264746 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307264746Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307264746Cleaning up temporary data from old regions at 1676307264761 (+15 ms)Cleaning up detritus from prior splits at 1676307264762 (+1 ms)Running coprocessor post-open hooks at 1676307264767 (+5 ms)Region opened successfully at 1676307264767 2023-02-13 16:54:24,778 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676307264650 2023-02-13 16:54:24,799 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 16:54:24,800 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 16:54:24,800 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34717,1676307263076, state=OPEN 2023-02-13 16:54:24,803 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 16:54:24,803 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 16:54:24,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 16:54:24,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34717,1676307263076 in 329 msec 2023-02-13 16:54:24,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 16:54:24,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 531 msec 2023-02-13 16:54:24,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 781 msec 2023-02-13 16:54:24,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676307263116, completionTime=-1 2023-02-13 16:54:24,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 16:54:24,821 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:24,884 DEBUG [hconnection-0x26bb8ba6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:24,887 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48866, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:24,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:24,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307324903 2023-02-13 16:54:24,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307384903 2023-02-13 16:54:24,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 82 msec 2023-02-13 16:54:24,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45273,1676307261815-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45273,1676307261815-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45273,1676307261815-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:45273, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 16:54:24,939 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 16:54:24,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 16:54:24,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 16:54:24,959 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:24,961 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:24,963 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:24,984 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/hbase/namespace/49a9767f24746f442cea05a1662ed767 2023-02-13 16:54:24,987 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/hbase/namespace/49a9767f24746f442cea05a1662ed767 empty. 2023-02-13 16:54:24,987 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/hbase/namespace/49a9767f24746f442cea05a1662ed767 2023-02-13 16:54:24,988 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 16:54:25,025 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:25,028 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 49a9767f24746f442cea05a1662ed767, NAME => 'hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:54:25,052 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:25,052 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 49a9767f24746f442cea05a1662ed767, disabling compactions & flushes 2023-02-13 16:54:25,052 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:54:25,052 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:54:25,053 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. after waiting 0 ms 2023-02-13 16:54:25,053 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:54:25,053 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:54:25,053 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 49a9767f24746f442cea05a1662ed767: Waiting for close lock at 1676307265052Disabling compacts and flushes for region at 1676307265052Disabling writes for close at 1676307265053 (+1 ms)Writing region close event to WAL at 1676307265053Closed at 1676307265053 2023-02-13 16:54:25,058 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:25,075 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676307265061"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307265061"}]},"ts":"1676307265061"} 2023-02-13 16:54:25,103 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:25,105 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:25,110 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307265106"}]},"ts":"1676307265106"} 2023-02-13 16:54:25,115 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 16:54:25,120 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:54:25,122 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:54:25,122 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:54:25,122 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:54:25,122 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:54:25,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=49a9767f24746f442cea05a1662ed767, ASSIGN}] 2023-02-13 16:54:25,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=49a9767f24746f442cea05a1662ed767, ASSIGN 2023-02-13 16:54:25,131 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=49a9767f24746f442cea05a1662ed767, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38051,1676307263041; forceNewPlan=false, retain=false 2023-02-13 16:54:25,283 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:54:25,284 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=49a9767f24746f442cea05a1662ed767, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:25,284 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676307265284"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307265284"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307265284"}]},"ts":"1676307265284"} 2023-02-13 16:54:25,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 49a9767f24746f442cea05a1662ed767, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:54:25,445 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:25,445 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 16:54:25,449 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45048, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 16:54:25,456 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:54:25,456 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 49a9767f24746f442cea05a1662ed767, NAME => 'hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:25,457 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:25,458 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 49a9767f24746f442cea05a1662ed767 2023-02-13 16:54:25,458 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:25,458 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 49a9767f24746f442cea05a1662ed767 2023-02-13 16:54:25,458 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 49a9767f24746f442cea05a1662ed767 2023-02-13 16:54:25,460 INFO [StoreOpener-49a9767f24746f442cea05a1662ed767-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 49a9767f24746f442cea05a1662ed767 2023-02-13 16:54:25,462 DEBUG [StoreOpener-49a9767f24746f442cea05a1662ed767-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767/info 2023-02-13 16:54:25,462 DEBUG [StoreOpener-49a9767f24746f442cea05a1662ed767-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767/info 2023-02-13 16:54:25,463 INFO [StoreOpener-49a9767f24746f442cea05a1662ed767-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 49a9767f24746f442cea05a1662ed767 columnFamilyName info 2023-02-13 16:54:25,464 INFO [StoreOpener-49a9767f24746f442cea05a1662ed767-1] regionserver.HStore(310): Store=49a9767f24746f442cea05a1662ed767/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:25,466 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767 2023-02-13 16:54:25,467 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767 2023-02-13 16:54:25,471 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 49a9767f24746f442cea05a1662ed767 2023-02-13 16:54:25,474 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:25,475 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 49a9767f24746f442cea05a1662ed767; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62708837, jitterRate=-0.06556551158428192}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:25,475 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 49a9767f24746f442cea05a1662ed767: Running coprocessor pre-open hook at 1676307265458Writing region info on filesystem at 1676307265458Initializing all the Stores at 1676307265459 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307265460 (+1 ms)Cleaning up temporary data from old regions at 1676307265467 (+7 ms)Cleaning up detritus from prior splits at 1676307265468 (+1 ms)Running coprocessor post-open hooks at 1676307265475 (+7 ms)Region opened successfully at 1676307265475 2023-02-13 16:54:25,477 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767., pid=6, masterSystemTime=1676307265444 2023-02-13 16:54:25,481 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:54:25,482 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:54:25,484 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=49a9767f24746f442cea05a1662ed767, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:25,484 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676307265483"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307265483"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307265483"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307265483"}]},"ts":"1676307265483"} 2023-02-13 16:54:25,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 16:54:25,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 49a9767f24746f442cea05a1662ed767, server=jenkins-hbase4.apache.org,38051,1676307263041 in 198 msec 2023-02-13 16:54:25,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 16:54:25,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=49a9767f24746f442cea05a1662ed767, ASSIGN in 365 msec 2023-02-13 16:54:25,497 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:25,497 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307265497"}]},"ts":"1676307265497"} 2023-02-13 16:54:25,499 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 16:54:25,502 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:25,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 552 msec 2023-02-13 16:54:25,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 16:54:25,563 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 16:54:25,563 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:54:25,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:25,593 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45054, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:25,608 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:25,628 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 16:54:25,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 34 msec 2023-02-13 16:54:25,641 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:25,652 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 16:54:25,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-02-13 16:54:25,668 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 16:54:25,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 16:54:25,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.541sec 2023-02-13 16:54:25,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 16:54:25,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 16:54:25,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 16:54:25,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45273,1676307261815-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 16:54:25,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45273,1676307261815-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 16:54:25,686 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 16:54:25,700 DEBUG [Listener at localhost/38547] zookeeper.ReadOnlyZKClient(139): Connect 0x0c8d5aec to 127.0.0.1:57428 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:25,706 DEBUG [Listener at localhost/38547] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ff608fa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:25,719 DEBUG [hconnection-0x4d838b63-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:25,730 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48874, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:25,741 INFO [Listener at localhost/38547] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:54:25,770 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=700, MaxFileDescriptor=60000, SystemLoadAverage=283, ProcessCount=168, AvailableMemoryMB=1634 2023-02-13 16:54:25,774 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:25,783 DEBUG [Listener at localhost/38547] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:54:25,786 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39290, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:54:25,798 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:54:25,798 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:54:25,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:54:25,805 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:25,807 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:25,809 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:25,812 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:25,812 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87 empty. 2023-02-13 16:54:25,814 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:25,815 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 16:54:25,867 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:25,869 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 13c13f79b91b6c01ec9b3edbb20ead87, NAME => 'testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:54:25,885 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:25,885 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 13c13f79b91b6c01ec9b3edbb20ead87, disabling compactions & flushes 2023-02-13 16:54:25,885 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:25,885 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:25,885 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. after waiting 0 ms 2023-02-13 16:54:25,885 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:25,886 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:25,886 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 13c13f79b91b6c01ec9b3edbb20ead87: Waiting for close lock at 1676307265885Disabling compacts and flushes for region at 1676307265885Disabling writes for close at 1676307265885Writing region close event to WAL at 1676307265886 (+1 ms)Closed at 1676307265886 2023-02-13 16:54:25,889 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:25,891 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676307265891"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307265891"}]},"ts":"1676307265891"} 2023-02-13 16:54:25,893 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:25,895 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:25,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307265895"}]},"ts":"1676307265895"} 2023-02-13 16:54:25,897 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-13 16:54:25,901 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:54:25,903 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:54:25,903 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:54:25,903 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:54:25,903 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:54:25,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=13c13f79b91b6c01ec9b3edbb20ead87, ASSIGN}] 2023-02-13 16:54:25,906 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=13c13f79b91b6c01ec9b3edbb20ead87, ASSIGN 2023-02-13 16:54:25,907 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=13c13f79b91b6c01ec9b3edbb20ead87, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38051,1676307263041; forceNewPlan=false, retain=false 2023-02-13 16:54:26,057 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:54:26,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=13c13f79b91b6c01ec9b3edbb20ead87, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,059 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676307266058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307266058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307266058"}]},"ts":"1676307266058"} 2023-02-13 16:54:26,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 13c13f79b91b6c01ec9b3edbb20ead87, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:54:26,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:26,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 13c13f79b91b6c01ec9b3edbb20ead87, NAME => 'testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:26,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:26,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:26,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:26,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:26,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:26,224 INFO [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:26,226 DEBUG [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1 2023-02-13 16:54:26,226 DEBUG [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1 2023-02-13 16:54:26,226 INFO [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 13c13f79b91b6c01ec9b3edbb20ead87 columnFamilyName cf1 2023-02-13 16:54:26,227 INFO [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] regionserver.HStore(310): Store=13c13f79b91b6c01ec9b3edbb20ead87/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:26,227 INFO [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:26,229 DEBUG [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2 2023-02-13 16:54:26,229 DEBUG [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2 2023-02-13 16:54:26,230 INFO [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 13c13f79b91b6c01ec9b3edbb20ead87 columnFamilyName cf2 2023-02-13 16:54:26,231 INFO [StoreOpener-13c13f79b91b6c01ec9b3edbb20ead87-1] regionserver.HStore(310): Store=13c13f79b91b6c01ec9b3edbb20ead87/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:26,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:26,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:26,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:26,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:26,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:26,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 13c13f79b91b6c01ec9b3edbb20ead87; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68549885, jitterRate=0.021472886204719543}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:26,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 13c13f79b91b6c01ec9b3edbb20ead87: Running coprocessor pre-open hook at 1676307266222Writing region info on filesystem at 1676307266222Initializing all the Stores at 1676307266223 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307266223Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307266224 (+1 ms)Cleaning up temporary data from old regions at 1676307266235 (+11 ms)Cleaning up detritus from prior splits at 1676307266235Running coprocessor post-open hooks at 1676307266243 (+8 ms)Region opened successfully at 1676307266243 2023-02-13 16:54:26,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87., pid=11, masterSystemTime=1676307266214 2023-02-13 16:54:26,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:26,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:26,248 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=13c13f79b91b6c01ec9b3edbb20ead87, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,249 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676307266248"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307266248"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307266248"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307266248"}]},"ts":"1676307266248"} 2023-02-13 16:54:26,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 16:54:26,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 13c13f79b91b6c01ec9b3edbb20ead87, server=jenkins-hbase4.apache.org,38051,1676307263041 in 191 msec 2023-02-13 16:54:26,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 16:54:26,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=13c13f79b91b6c01ec9b3edbb20ead87, ASSIGN in 353 msec 2023-02-13 16:54:26,260 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:26,261 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307266261"}]},"ts":"1676307266261"} 2023-02-13 16:54:26,263 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-13 16:54:26,265 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:26,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 463 msec 2023-02-13 16:54:26,309 DEBUG [Listener at localhost/38547] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:26,313 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45068, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:26,331 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:26,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,338 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:26,338 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:26,339 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 16:54:26,339 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 16:54:26,339 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,340 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,341 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:26,341 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:26,341 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:26,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:26,341 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:26,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:26,341 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,342 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,345 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:26,345 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:26,345 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:26,347 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:26,347 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:26,347 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:26,347 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:26,347 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:26,347 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:26,347 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:26,348 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. started... 2023-02-13 16:54:26,349 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13c13f79b91b6c01ec9b3edbb20ead87 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 16:54:26,349 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,349 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,350 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,350 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:26,350 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,350 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:26,421 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/317314500b084b0fbc0843178fd8dfa1 2023-02-13 16:54:26,884 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/a0669c81f92f4316b8f40da3b6c33dbd 2023-02-13 16:54:26,894 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/317314500b084b0fbc0843178fd8dfa1 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1 2023-02-13 16:54:26,904 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:54:26,906 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/a0669c81f92f4316b8f40da3b6c33dbd as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd 2023-02-13 16:54:26,916 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:54:26,918 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 13c13f79b91b6c01ec9b3edbb20ead87 in 569ms, sequenceid=6, compaction requested=false 2023-02-13 16:54:26,919 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:26,922 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13c13f79b91b6c01ec9b3edbb20ead87: 2023-02-13 16:54:26,922 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:26,922 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:26,923 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:26,923 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,923 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:26,923 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:26,926 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,926 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:26,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:26,927 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,927 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:26,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:26,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:26,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:26,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:26,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:26,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 16:54:26,930 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:26,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32829a9c[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:26,930 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:26,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,939 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:26,939 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:26,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,939 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:26,939 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:26,939 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:26,940 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:26,939 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:26,940 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:26,940 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:26,945 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:26,945 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:26,945 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:26,947 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,947 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:26,947 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:26,947 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:26,947 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:26,948 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:26,948 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:26,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:26,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:26,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:26,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:54:26,953 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 16:54:26,954 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:26,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:26,954 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:26,954 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,954 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:26,956 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,956 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,956 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,956 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:26,956 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:26,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,956 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,956 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:26,957 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:26,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:26,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:26,958 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,958 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:26,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:26,961 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,962 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,962 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:26,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:26,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:26,975 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:26,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:26,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:26,975 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:26,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:26,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:26,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:26,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:26,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:26,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:26,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:26,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:26,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:26,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:26,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,977 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,977 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:26,977 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,977 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:26,977 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:26,977 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,977 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:26,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:26,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:26,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:26,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:26,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:26,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:26,982 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:26,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:26,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:26,986 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:26,986 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:27,086 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:27,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:27,117 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:27,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:27,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,122 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:27,122 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,123 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 16:54:27,124 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 16:54:27,124 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,124 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,126 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,126 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,126 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,126 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,127 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,127 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,128 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,129 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,129 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,129 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,131 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,131 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,131 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,131 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:27,132 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:27,133 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. started... 2023-02-13 16:54:27,133 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13c13f79b91b6c01ec9b3edbb20ead87 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:27,135 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,135 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,135 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,136 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,136 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,136 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,160 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d 2023-02-13 16:54:27,187 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/65aae0558ae048a5b5475aea1f66b32d 2023-02-13 16:54:27,196 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d 2023-02-13 16:54:27,203 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:54:27,204 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/65aae0558ae048a5b5475aea1f66b32d as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d 2023-02-13 16:54:27,213 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:54:27,214 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13c13f79b91b6c01ec9b3edbb20ead87 in 81ms, sequenceid=19, compaction requested=false 2023-02-13 16:54:27,214 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13c13f79b91b6c01ec9b3edbb20ead87: 2023-02-13 16:54:27,214 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:27,214 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:27,216 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:27,216 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,216 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,216 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,218 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,218 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,219 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,219 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 16:54:27,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47f53d05[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:27,221 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:27,221 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,222 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,223 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,223 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,223 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,223 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,223 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:27,223 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,223 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,223 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,223 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:27,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,223 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:27,223 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,223 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,223 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:27,226 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,226 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,226 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,227 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,227 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,227 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,227 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,227 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,227 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,228 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:54:27,233 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:27,233 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 16:54:27,233 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:27,233 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:27,233 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,233 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:27,235 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,235 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,235 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,235 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,235 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,235 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,235 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,236 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,237 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,238 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,240 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,240 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,241 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,251 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,251 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,251 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,251 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,251 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,252 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,252 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:27,252 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,252 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,251 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,251 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,253 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,252 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,252 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:27,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,253 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,253 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,253 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,253 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,253 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,253 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,254 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:27,254 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:27,354 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:27,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:27,379 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:27,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:27,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,382 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:27,382 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,383 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 16:54:27,383 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 16:54:27,384 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,384 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,385 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,385 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,385 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,385 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,386 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,386 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,388 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,388 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,388 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,389 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,389 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,389 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,389 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,389 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,389 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,389 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,389 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,389 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,389 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,389 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,390 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,390 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,390 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,390 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,390 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:27,391 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:27,391 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:27,391 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. started... 2023-02-13 16:54:27,391 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13c13f79b91b6c01ec9b3edbb20ead87 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:27,393 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,394 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,394 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,394 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,394 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,394 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,409 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/04feb39ef4524fff89bce04d2ff24411 2023-02-13 16:54:27,433 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/6271c1b16ce7414ea073676efb8b484e 2023-02-13 16:54:27,441 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/04feb39ef4524fff89bce04d2ff24411 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411 2023-02-13 16:54:27,448 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:54:27,449 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/6271c1b16ce7414ea073676efb8b484e as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e 2023-02-13 16:54:27,456 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:54:27,457 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13c13f79b91b6c01ec9b3edbb20ead87 in 66ms, sequenceid=32, compaction requested=true 2023-02-13 16:54:27,457 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13c13f79b91b6c01ec9b3edbb20ead87: 2023-02-13 16:54:27,457 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:27,457 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:27,458 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:27,458 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,458 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,458 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,461 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,461 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,461 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,461 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 16:54:27,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@363af500[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:27,464 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:27,464 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,465 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,465 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,465 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,465 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,465 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:27,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,465 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,465 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,465 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,466 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,466 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,466 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:27,465 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,466 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:27,466 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,469 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,470 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,470 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:54:27,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:27,475 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 16:54:27,475 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:27,475 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:27,475 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,475 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:27,477 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,477 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,478 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,477 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,477 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,478 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,478 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,478 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,479 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,480 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,488 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,488 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,488 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,492 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,492 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,492 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,493 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:27,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,493 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:27,493 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,494 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,494 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,494 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,494 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,494 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,494 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,594 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:27,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:27,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:27,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:27,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,619 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:27,619 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,620 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 16:54:27,620 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 16:54:27,621 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,621 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,622 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,622 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,622 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,622 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,623 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,623 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:27,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,624 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,624 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,625 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,625 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,625 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,625 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,626 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,625 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,626 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,626 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,626 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,626 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,627 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:27,627 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:27,627 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,627 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:27,627 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,627 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. started... 2023-02-13 16:54:27,627 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,628 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13c13f79b91b6c01ec9b3edbb20ead87 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:27,629 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,629 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,629 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,631 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,631 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,631 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,650 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/1f8a17c22d07426ea4b90d96b99bbee5 2023-02-13 16:54:27,674 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/07e02ed32994456cb72efcfb4a36d039 2023-02-13 16:54:27,682 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/1f8a17c22d07426ea4b90d96b99bbee5 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5 2023-02-13 16:54:27,690 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:54:27,691 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/07e02ed32994456cb72efcfb4a36d039 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039 2023-02-13 16:54:27,699 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:54:27,700 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13c13f79b91b6c01ec9b3edbb20ead87 in 73ms, sequenceid=45, compaction requested=true 2023-02-13 16:54:27,701 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13c13f79b91b6c01ec9b3edbb20ead87: 2023-02-13 16:54:27,701 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,706 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,706 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,706 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,706 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 16:54:27,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@216abc12[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:27,708 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:27,709 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,710 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,710 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,710 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,710 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,710 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:27,710 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:27,710 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,710 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:27,710 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,710 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:27,712 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,712 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,712 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,713 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,713 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,713 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,713 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,713 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,714 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,714 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:54:27,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:27,718 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 16:54:27,718 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:27,718 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:27,718 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,718 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:27,720 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,720 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,720 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,720 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,720 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,720 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,720 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,721 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,721 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,722 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,724 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,724 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,725 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,728 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,728 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,728 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,728 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,728 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,728 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,728 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,728 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,728 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,728 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:27,729 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:27,728 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,729 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,729 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,729 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:27,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,729 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,730 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,730 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:27,730 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,730 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,730 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,730 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,830 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:27,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:27,852 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:27,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:27,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,855 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:27,855 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,856 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 16:54:27,856 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 16:54:27,856 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,856 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,859 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,859 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,859 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,859 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,859 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,859 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:27,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,860 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,860 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,860 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,860 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:27,861 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,864 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,864 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,864 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:27,864 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,864 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:27,864 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:27,864 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. started... 2023-02-13 16:54:27,864 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,864 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,864 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:27,866 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,866 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:27,866 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13c13f79b91b6c01ec9b3edbb20ead87 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:27,866 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,866 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,868 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,869 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,869 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,869 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,869 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,869 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,886 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/73fbfcf85bf94e959d070ec78e3f0f13 2023-02-13 16:54:27,910 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/41edc9dcef45425d89f150b5cec3f5ca 2023-02-13 16:54:27,918 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/73fbfcf85bf94e959d070ec78e3f0f13 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/73fbfcf85bf94e959d070ec78e3f0f13 2023-02-13 16:54:27,925 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/73fbfcf85bf94e959d070ec78e3f0f13, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:54:27,926 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/41edc9dcef45425d89f150b5cec3f5ca as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/41edc9dcef45425d89f150b5cec3f5ca 2023-02-13 16:54:27,933 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/41edc9dcef45425d89f150b5cec3f5ca, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:54:27,934 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13c13f79b91b6c01ec9b3edbb20ead87 in 68ms, sequenceid=58, compaction requested=true 2023-02-13 16:54:27,934 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13c13f79b91b6c01ec9b3edbb20ead87: 2023-02-13 16:54:27,934 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:27,934 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:27,935 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:27,935 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,935 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:27,935 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:27,936 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,936 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,937 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,937 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 16:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71a94d9[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:27,939 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:27,939 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,941 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,941 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:27,941 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,941 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,941 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,941 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:27,942 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:27,942 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:27,942 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:27,942 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:27,943 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,943 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,943 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,943 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,944 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,944 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,944 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:27,944 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,945 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,945 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:27,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:54:27,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:27,949 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 16:54:27,950 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:27,950 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:27,950 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,950 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:27,951 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,951 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,951 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,951 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,951 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,951 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,951 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,952 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,953 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,953 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,955 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,956 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,956 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,964 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,963 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:27,964 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:27,963 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,964 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:27,965 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:27,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,065 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:28,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:28,088 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:28,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:28,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,091 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:28,091 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:28,092 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 16:54:28,092 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 16:54:28,092 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,092 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,094 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,094 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:28,094 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:28,094 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:28,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:28,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:28,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:28,094 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,094 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:28,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:28,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:28,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:28,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,096 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,096 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,096 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:28,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:28,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 16:54:28,096 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,097 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:28,097 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:28,097 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,097 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:28,097 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 16:54:28,098 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:28,098 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:28,097 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:28,097 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:28,098 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. started... 2023-02-13 16:54:28,098 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 16:54:28,099 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:28,099 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:28,098 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:28,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:28,099 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13c13f79b91b6c01ec9b3edbb20ead87 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:28,100 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,100 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:28,116 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/cf1ed060c9844ab997f8b7cc03c90911 2023-02-13 16:54:28,137 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/5b2d72ee795a47c780dc300dd8a1dcb2 2023-02-13 16:54:28,144 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/cf1ed060c9844ab997f8b7cc03c90911 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/cf1ed060c9844ab997f8b7cc03c90911 2023-02-13 16:54:28,151 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/cf1ed060c9844ab997f8b7cc03c90911, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:54:28,152 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/5b2d72ee795a47c780dc300dd8a1dcb2 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/5b2d72ee795a47c780dc300dd8a1dcb2 2023-02-13 16:54:28,158 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/5b2d72ee795a47c780dc300dd8a1dcb2, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:54:28,159 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13c13f79b91b6c01ec9b3edbb20ead87 in 60ms, sequenceid=71, compaction requested=true 2023-02-13 16:54:28,159 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13c13f79b91b6c01ec9b3edbb20ead87: 2023-02-13 16:54:28,160 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:28,160 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:28,161 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:28,161 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,161 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 16:54:28,161 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 16:54:28,162 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,162 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:28,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:28,163 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,163 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:28,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:28,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:28,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:28,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:28,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:28,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 16:54:28,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69699c2a[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:28,165 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:28,165 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,172 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,172 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,172 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,172 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,172 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:28,172 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:28,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,172 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:28,173 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:28,173 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:28,173 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:28,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,173 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:28,173 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 16:54:28,173 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 16:54:28,173 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:28,174 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:28,174 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:28,174 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:28,175 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,175 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:28,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:28,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:28,175 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:28,175 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 16:54:28,175 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:28,175 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:28,176 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 16:54:28,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:28,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:28,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:28,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:28,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:28,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:28,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:28,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:54:28,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:28,180 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 16:54:28,180 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:28,180 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:28,180 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,180 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:28,181 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,181 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,182 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,181 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,182 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:28,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,181 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:28,181 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:28,182 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:28,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:28,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:28,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:28,182 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:28,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:28,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:28,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,183 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:28,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:28,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,185 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:28,185 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:28,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:28,185 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:28,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:28,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:28,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:28,189 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:28,189 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,189 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:28,189 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:28,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:28,189 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:28,189 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:28,189 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:28,189 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:28,189 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:28,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter because node does not exist (not an error) 2023-02-13 16:54:28,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:28,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:28,190 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:28,189 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 16:54:28,190 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:28,189 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:28,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:28,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:28,190 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:28,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,190 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:28,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:28,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,190 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:28,191 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,190 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:28,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:28,191 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:28,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,191 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,191 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,191 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 16:54:28,291 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:28,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:28,316 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,323 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:28,324 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:28,324 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:28,324 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:28,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=13c13f79b91b6c01ec9b3edbb20ead87, UNASSIGN}] 2023-02-13 16:54:28,332 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=13c13f79b91b6c01ec9b3edbb20ead87, UNASSIGN 2023-02-13 16:54:28,333 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=13c13f79b91b6c01ec9b3edbb20ead87, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:28,333 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676307268333"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307268333"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307268333"}]},"ts":"1676307268333"} 2023-02-13 16:54:28,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 13c13f79b91b6c01ec9b3edbb20ead87, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:54:28,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 13c13f79b91b6c01ec9b3edbb20ead87, disabling compactions & flushes 2023-02-13 16:54:28,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:28,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:28,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. after waiting 0 ms 2023-02-13 16:54:28,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:28,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 13c13f79b91b6c01ec9b3edbb20ead87 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 16:54:28,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/714a8f4733484303babdf59a602d3b20 2023-02-13 16:54:28,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/4376f3f7c56647edabc6bc093e2453d2 2023-02-13 16:54:28,546 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf1/714a8f4733484303babdf59a602d3b20 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/714a8f4733484303babdf59a602d3b20 2023-02-13 16:54:28,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/714a8f4733484303babdf59a602d3b20, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:54:28,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.tmp/cf2/4376f3f7c56647edabc6bc093e2453d2 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/4376f3f7c56647edabc6bc093e2453d2 2023-02-13 16:54:28,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/4376f3f7c56647edabc6bc093e2453d2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:54:28,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 13c13f79b91b6c01ec9b3edbb20ead87 in 68ms, sequenceid=82, compaction requested=true 2023-02-13 16:54:28,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 16:54:28,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:28,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. 2023-02-13 16:54:28,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 13c13f79b91b6c01ec9b3edbb20ead87: Waiting for close lock at 1676307268494Running coprocessor pre-close hooks at 1676307268494Disabling compacts and flushes for region at 1676307268494Disabling writes for close at 1676307268495 (+1 ms)Obtaining lock to block concurrent updates at 1676307268495Preparing flush snapshotting stores in 13c13f79b91b6c01ec9b3edbb20ead87 at 1676307268495Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676307268495Flushing stores of testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. at 1676307268496 (+1 ms)Flushing 13c13f79b91b6c01ec9b3edbb20ead87/cf1: creating writer at 1676307268496Flushing 13c13f79b91b6c01ec9b3edbb20ead87/cf1: appending metadata at 1676307268499 (+3 ms)Flushing 13c13f79b91b6c01ec9b3edbb20ead87/cf1: closing flushed file at 1676307268499Flushing 13c13f79b91b6c01ec9b3edbb20ead87/cf2: creating writer at 1676307268521 (+22 ms)Flushing 13c13f79b91b6c01ec9b3edbb20ead87/cf2: appending metadata at 1676307268524 (+3 ms)Flushing 13c13f79b91b6c01ec9b3edbb20ead87/cf2: closing flushed file at 1676307268524Flushing 13c13f79b91b6c01ec9b3edbb20ead87/cf1: reopening flushed file at 1676307268547 (+23 ms)Flushing 13c13f79b91b6c01ec9b3edbb20ead87/cf2: reopening flushed file at 1676307268556 (+9 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 13c13f79b91b6c01ec9b3edbb20ead87 in 68ms, sequenceid=82, compaction requested=true at 1676307268563 (+7 ms)Writing region close event to WAL at 1676307268578 (+15 ms)Running coprocessor post-close hooks at 1676307268583 (+5 ms)Closed at 1676307268584 (+1 ms) 2023-02-13 16:54:28,587 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,587 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=13c13f79b91b6c01ec9b3edbb20ead87, regionState=CLOSED 2023-02-13 16:54:28,588 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676307268587"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307268587"}]},"ts":"1676307268587"} 2023-02-13 16:54:28,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 16:54:28,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 13c13f79b91b6c01ec9b3edbb20ead87, server=jenkins-hbase4.apache.org,38051,1676307263041 in 255 msec 2023-02-13 16:54:28,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-13 16:54:28,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=13c13f79b91b6c01ec9b3edbb20ead87, UNASSIGN in 262 msec 2023-02-13 16:54:28,614 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=13c13f79b91b6c01ec9b3edbb20ead87, threads=14 2023-02-13 16:54:28,614 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,614 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/41edc9dcef45425d89f150b5cec3f5ca for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,615 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/4376f3f7c56647edabc6bc093e2453d2 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,615 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/5b2d72ee795a47c780dc300dd8a1dcb2 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,615 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,615 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,615 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,615 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,615 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,615 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,616 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/714a8f4733484303babdf59a602d3b20 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,616 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/73fbfcf85bf94e959d070ec78e3f0f13 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,616 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,622 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/cf1ed060c9844ab997f8b7cc03c90911 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,649 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/714a8f4733484303babdf59a602d3b20, top=true 2023-02-13 16:54:28,666 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/cf1ed060c9844ab997f8b7cc03c90911, top=true 2023-02-13 16:54:28,674 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/4376f3f7c56647edabc6bc093e2453d2, top=true 2023-02-13 16:54:28,674 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/5b2d72ee795a47c780dc300dd8a1dcb2, top=true 2023-02-13 16:54:28,681 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/41edc9dcef45425d89f150b5cec3f5ca, top=true 2023-02-13 16:54:28,714 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.splits/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-714a8f4733484303babdf59a602d3b20 for child: 8b86be70872f3798a9e95e2f1e613a73, parent: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,714 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/714a8f4733484303babdf59a602d3b20 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,716 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/73fbfcf85bf94e959d070ec78e3f0f13, top=true 2023-02-13 16:54:28,727 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.splits/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-4376f3f7c56647edabc6bc093e2453d2 for child: 8b86be70872f3798a9e95e2f1e613a73, parent: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,727 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/4376f3f7c56647edabc6bc093e2453d2 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,731 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.splits/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-5b2d72ee795a47c780dc300dd8a1dcb2 for child: 8b86be70872f3798a9e95e2f1e613a73, parent: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,731 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/5b2d72ee795a47c780dc300dd8a1dcb2 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,732 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.splits/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-cf1ed060c9844ab997f8b7cc03c90911 for child: 8b86be70872f3798a9e95e2f1e613a73, parent: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,732 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/cf1ed060c9844ab997f8b7cc03c90911 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,732 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.splits/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-41edc9dcef45425d89f150b5cec3f5ca for child: 8b86be70872f3798a9e95e2f1e613a73, parent: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,733 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/41edc9dcef45425d89f150b5cec3f5ca for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,737 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/.splits/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-73fbfcf85bf94e959d070ec78e3f0f13 for child: 8b86be70872f3798a9e95e2f1e613a73, parent: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,737 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/73fbfcf85bf94e959d070ec78e3f0f13 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,812 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,816 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,816 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,822 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,823 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,825 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:28,826 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5 for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:29,144 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd for region: 13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:29,144 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 13c13f79b91b6c01ec9b3edbb20ead87 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-13 16:54:29,193 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:54:29,195 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:54:29,198 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676307269198"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676307269198"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676307269198"}]},"ts":"1676307269198"} 2023-02-13 16:54:29,198 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307269198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307269198"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307269198"}]},"ts":"1676307269198"} 2023-02-13 16:54:29,198 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307269198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307269198"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307269198"}]},"ts":"1676307269198"} 2023-02-13 16:54:29,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fea250aff2133b2f109bfd4e8765243e, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=8b86be70872f3798a9e95e2f1e613a73, ASSIGN}] 2023-02-13 16:54:29,247 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=8b86be70872f3798a9e95e2f1e613a73, ASSIGN 2023-02-13 16:54:29,247 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=8b86be70872f3798a9e95e2f1e613a73, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38051,1676307263041; forceNewPlan=false, retain=false 2023-02-13 16:54:29,248 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fea250aff2133b2f109bfd4e8765243e, ASSIGN 2023-02-13 16:54:29,249 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fea250aff2133b2f109bfd4e8765243e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38051,1676307263041; forceNewPlan=false, retain=false 2023-02-13 16:54:29,398 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 16:54:29,399 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=8b86be70872f3798a9e95e2f1e613a73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:29,399 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307269399"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307269399"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307269399"}]},"ts":"1676307269399"} 2023-02-13 16:54:29,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 8b86be70872f3798a9e95e2f1e613a73, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:54:29,406 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=fea250aff2133b2f109bfd4e8765243e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:29,406 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307269406"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307269406"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307269406"}]},"ts":"1676307269406"} 2023-02-13 16:54:29,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure fea250aff2133b2f109bfd4e8765243e, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:54:29,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:29,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8b86be70872f3798a9e95e2f1e613a73, NAME => 'testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.', STARTKEY => '26', ENDKEY => ''} 2023-02-13 16:54:29,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:29,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:29,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:29,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:29,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:29,562 INFO [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:29,563 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1 2023-02-13 16:54:29,563 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1 2023-02-13 16:54:29,564 INFO [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8b86be70872f3798a9e95e2f1e613a73 columnFamilyName cf1 2023-02-13 16:54:29,599 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411-top 2023-02-13 16:54:29,606 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5-top 2023-02-13 16:54:29,613 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1-top 2023-02-13 16:54:29,620 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d-top 2023-02-13 16:54:29,626 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-714a8f4733484303babdf59a602d3b20 2023-02-13 16:54:29,631 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-73fbfcf85bf94e959d070ec78e3f0f13 2023-02-13 16:54:29,635 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-cf1ed060c9844ab997f8b7cc03c90911 2023-02-13 16:54:29,636 INFO [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(310): Store=8b86be70872f3798a9e95e2f1e613a73/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:29,636 INFO [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:29,637 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2 2023-02-13 16:54:29,637 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2 2023-02-13 16:54:29,638 INFO [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8b86be70872f3798a9e95e2f1e613a73 columnFamilyName cf2 2023-02-13 16:54:29,653 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039-top 2023-02-13 16:54:29,658 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e-top 2023-02-13 16:54:29,663 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d-top 2023-02-13 16:54:29,668 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd-top 2023-02-13 16:54:29,673 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-41edc9dcef45425d89f150b5cec3f5ca 2023-02-13 16:54:29,677 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-4376f3f7c56647edabc6bc093e2453d2 2023-02-13 16:54:29,682 DEBUG [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-5b2d72ee795a47c780dc300dd8a1dcb2 2023-02-13 16:54:29,682 INFO [StoreOpener-8b86be70872f3798a9e95e2f1e613a73-1] regionserver.HStore(310): Store=8b86be70872f3798a9e95e2f1e613a73/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:29,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:29,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:29,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:29,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:29,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8b86be70872f3798a9e95e2f1e613a73; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68521648, jitterRate=0.021052122116088867}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:29,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8b86be70872f3798a9e95e2f1e613a73: Running coprocessor pre-open hook at 1676307269560Writing region info on filesystem at 1676307269560Initializing all the Stores at 1676307269561 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307269561Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307269562 (+1 ms)Cleaning up temporary data from old regions at 1676307269686 (+124 ms)Cleaning up detritus from prior splits at 1676307269686Running coprocessor post-open hooks at 1676307269689 (+3 ms)Region opened successfully at 1676307269690 (+1 ms) 2023-02-13 16:54:29,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73., pid=17, masterSystemTime=1676307269554 2023-02-13 16:54:29,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:29,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:54:29,695 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:29,695 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:29,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:29,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:29,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:29,698 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:29,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fea250aff2133b2f109bfd4e8765243e, NAME => 'testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.', STARTKEY => '', ENDKEY => '26'} 2023-02-13 16:54:29,698 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:29,698 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1912): 8b86be70872f3798a9e95e2f1e613a73/cf2 is initiating minor compaction (all files) 2023-02-13 16:54:29,698 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1912): 8b86be70872f3798a9e95e2f1e613a73/cf1 is initiating minor compaction (all files) 2023-02-13 16:54:29,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:29,699 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=8b86be70872f3798a9e95e2f1e613a73, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:29,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:29,699 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8b86be70872f3798a9e95e2f1e613a73/cf2 in testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:29,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:29,699 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8b86be70872f3798a9e95e2f1e613a73/cf1 in testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:29,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:29,699 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307269698"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307269698"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307269698"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307269698"}]},"ts":"1676307269698"} 2023-02-13 16:54:29,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:29,699 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-41edc9dcef45425d89f150b5cec3f5ca, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-5b2d72ee795a47c780dc300dd8a1dcb2, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-4376f3f7c56647edabc6bc093e2453d2] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/.tmp, totalSize=34.9 K 2023-02-13 16:54:29,700 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-73fbfcf85bf94e959d070ec78e3f0f13, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-cf1ed060c9844ab997f8b7cc03c90911, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-714a8f4733484303babdf59a602d3b20] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/.tmp, totalSize=34.9 K 2023-02-13 16:54:29,702 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307266313 2023-02-13 16:54:29,702 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307266313 2023-02-13 16:54:29,704 INFO [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:29,705 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307267094 2023-02-13 16:54:29,705 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting 65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307267094 2023-02-13 16:54:29,706 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307267356 2023-02-13 16:54:29,706 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting 6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307267356 2023-02-13 16:54:29,706 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1 2023-02-13 16:54:29,707 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1 2023-02-13 16:54:29,707 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676307267596 2023-02-13 16:54:29,707 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting 07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676307267596 2023-02-13 16:54:29,707 INFO [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fea250aff2133b2f109bfd4e8765243e columnFamilyName cf1 2023-02-13 16:54:29,707 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-41edc9dcef45425d89f150b5cec3f5ca, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307267832 2023-02-13 16:54:29,708 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-73fbfcf85bf94e959d070ec78e3f0f13, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307267832 2023-02-13 16:54:29,708 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-5b2d72ee795a47c780dc300dd8a1dcb2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307268068 2023-02-13 16:54:29,709 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-cf1ed060c9844ab997f8b7cc03c90911, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307268068 2023-02-13 16:54:29,710 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-4376f3f7c56647edabc6bc093e2453d2, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307268293 2023-02-13 16:54:29,711 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-714a8f4733484303babdf59a602d3b20, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307268293 2023-02-13 16:54:29,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-13 16:54:29,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 8b86be70872f3798a9e95e2f1e613a73, server=jenkins-hbase4.apache.org,38051,1676307263041 in 303 msec 2023-02-13 16:54:29,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=8b86be70872f3798a9e95e2f1e613a73, ASSIGN in 467 msec 2023-02-13 16:54:29,726 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411-bottom 2023-02-13 16:54:29,735 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5-bottom 2023-02-13 16:54:29,742 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8b86be70872f3798a9e95e2f1e613a73#cf1#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:29,744 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] throttle.PressureAwareThroughputController(145): 8b86be70872f3798a9e95e2f1e613a73#cf2#compaction#15 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:29,746 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1-bottom 2023-02-13 16:54:29,754 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d-bottom 2023-02-13 16:54:29,756 INFO [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(310): Store=fea250aff2133b2f109bfd4e8765243e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:29,756 INFO [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:29,757 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2 2023-02-13 16:54:29,757 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2 2023-02-13 16:54:29,758 INFO [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fea250aff2133b2f109bfd4e8765243e columnFamilyName cf2 2023-02-13 16:54:29,799 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/.tmp/cf1/2d1742f6102f44f284b2694b0f51ad06 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/2d1742f6102f44f284b2694b0f51ad06 2023-02-13 16:54:29,803 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/.tmp/cf2/194feedc4ba34676aed7bc60ef7237b2 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/194feedc4ba34676aed7bc60ef7237b2 2023-02-13 16:54:29,807 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039-bottom 2023-02-13 16:54:29,812 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e-bottom 2023-02-13 16:54:29,817 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d-bottom 2023-02-13 16:54:29,817 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8b86be70872f3798a9e95e2f1e613a73/cf1 of 8b86be70872f3798a9e95e2f1e613a73 into 2d1742f6102f44f284b2694b0f51ad06(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:29,817 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8b86be70872f3798a9e95e2f1e613a73/cf2 of 8b86be70872f3798a9e95e2f1e613a73 into 194feedc4ba34676aed7bc60ef7237b2(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:29,818 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8b86be70872f3798a9e95e2f1e613a73: 2023-02-13 16:54:29,818 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8b86be70872f3798a9e95e2f1e613a73: 2023-02-13 16:54:29,818 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73., storeName=8b86be70872f3798a9e95e2f1e613a73/cf1, priority=9, startTime=1676307269693; duration=0sec 2023-02-13 16:54:29,818 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73., storeName=8b86be70872f3798a9e95e2f1e613a73/cf2, priority=9, startTime=1676307269693; duration=0sec 2023-02-13 16:54:29,819 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:29,819 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:29,822 DEBUG [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd-bottom 2023-02-13 16:54:29,822 INFO [StoreOpener-fea250aff2133b2f109bfd4e8765243e-1] regionserver.HStore(310): Store=fea250aff2133b2f109bfd4e8765243e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:29,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:29,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:29,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:29,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:29,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened fea250aff2133b2f109bfd4e8765243e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65440574, jitterRate=-0.024859458208084106}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:29,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for fea250aff2133b2f109bfd4e8765243e: Running coprocessor pre-open hook at 1676307269699Writing region info on filesystem at 1676307269699Initializing all the Stores at 1676307269702 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307269702Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307269704 (+2 ms)Cleaning up temporary data from old regions at 1676307269827 (+123 ms)Cleaning up detritus from prior splits at 1676307269827Running coprocessor post-open hooks at 1676307269831 (+4 ms)Region opened successfully at 1676307269831 2023-02-13 16:54:29,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e., pid=18, masterSystemTime=1676307269554 2023-02-13 16:54:29,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:29,833 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 16:54:29,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:29,835 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 16:54:29,835 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:29,835 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1912): fea250aff2133b2f109bfd4e8765243e/cf1 is initiating minor compaction (all files) 2023-02-13 16:54:29,835 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:29,835 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of fea250aff2133b2f109bfd4e8765243e/cf1 in testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:29,835 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1912): fea250aff2133b2f109bfd4e8765243e/cf2 is initiating minor compaction (all files) 2023-02-13 16:54:29,835 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegion(2258): Starting compaction of fea250aff2133b2f109bfd4e8765243e/cf2 in testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:29,835 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/.tmp, totalSize=19.9 K 2023-02-13 16:54:29,836 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/.tmp, totalSize=19.9 K 2023-02-13 16:54:29,836 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307266313 2023-02-13 16:54:29,837 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307266313 2023-02-13 16:54:29,837 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307267094 2023-02-13 16:54:29,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:29,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:29,838 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting 65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307267094 2023-02-13 16:54:29,838 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307267356 2023-02-13 16:54:29,838 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=fea250aff2133b2f109bfd4e8765243e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:29,838 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting 6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307267356 2023-02-13 16:54:29,838 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307269838"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307269838"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307269838"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307269838"}]},"ts":"1676307269838"} 2023-02-13 16:54:29,839 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307267596 2023-02-13 16:54:29,839 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting 07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307267596 2023-02-13 16:54:29,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-13 16:54:29,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure fea250aff2133b2f109bfd4e8765243e, server=jenkins-hbase4.apache.org,38051,1676307263041 in 434 msec 2023-02-13 16:54:29,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-13 16:54:29,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fea250aff2133b2f109bfd4e8765243e, ASSIGN in 600 msec 2023-02-13 16:54:29,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 in 1.5310 sec 2023-02-13 16:54:29,857 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] throttle.PressureAwareThroughputController(145): fea250aff2133b2f109bfd4e8765243e#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:29,857 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] throttle.PressureAwareThroughputController(145): fea250aff2133b2f109bfd4e8765243e#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:29,876 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/.tmp/cf1/99eca88161864a1f92b5742e974c7fa1 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/99eca88161864a1f92b5742e974c7fa1 2023-02-13 16:54:29,876 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/.tmp/cf2/a7358a66fa4c41ccb630e1072c595158 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/a7358a66fa4c41ccb630e1072c595158 2023-02-13 16:54:29,884 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in fea250aff2133b2f109bfd4e8765243e/cf1 of fea250aff2133b2f109bfd4e8765243e into 99eca88161864a1f92b5742e974c7fa1(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:29,884 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in fea250aff2133b2f109bfd4e8765243e/cf2 of fea250aff2133b2f109bfd4e8765243e into a7358a66fa4c41ccb630e1072c595158(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:29,884 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for fea250aff2133b2f109bfd4e8765243e: 2023-02-13 16:54:29,884 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for fea250aff2133b2f109bfd4e8765243e: 2023-02-13 16:54:29,884 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e., storeName=fea250aff2133b2f109bfd4e8765243e/cf1, priority=12, startTime=1676307269832; duration=0sec 2023-02-13 16:54:29,884 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e., storeName=fea250aff2133b2f109bfd4e8765243e/cf2, priority=12, startTime=1676307269833; duration=0sec 2023-02-13 16:54:29,885 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:29,885 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:30,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38051] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:45068 deadline: 1676307330075, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87. is not online on jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:30,209 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 16:54:30,220 INFO [Listener at localhost/38547] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 16:54:30,223 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 16:54:30,262 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:30,272 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,276 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,287 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307270287"}]},"ts":"1676307270287"} 2023-02-13 16:54:30,289 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-13 16:54:30,291 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-13 16:54:30,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fea250aff2133b2f109bfd4e8765243e, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=8b86be70872f3798a9e95e2f1e613a73, UNASSIGN}] 2023-02-13 16:54:30,294 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=8b86be70872f3798a9e95e2f1e613a73, UNASSIGN 2023-02-13 16:54:30,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 16:54:30,295 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=8b86be70872f3798a9e95e2f1e613a73, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:30,295 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307270295"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307270295"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307270295"}]},"ts":"1676307270295"} 2023-02-13 16:54:30,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 8b86be70872f3798a9e95e2f1e613a73, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:54:30,299 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:30,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fea250aff2133b2f109bfd4e8765243e, UNASSIGN 2023-02-13 16:54:30,307 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fea250aff2133b2f109bfd4e8765243e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:30,308 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307270307"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307270307"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307270307"}]},"ts":"1676307270307"} 2023-02-13 16:54:30,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure fea250aff2133b2f109bfd4e8765243e, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:54:30,359 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 16:54:30,360 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 16:54:30,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 16:54:30,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:30,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8b86be70872f3798a9e95e2f1e613a73, disabling compactions & flushes 2023-02-13 16:54:30,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:30,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:30,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. after waiting 0 ms 2023-02-13 16:54:30,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:30,467 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-73fbfcf85bf94e959d070ec78e3f0f13, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-cf1ed060c9844ab997f8b7cc03c90911, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-714a8f4733484303babdf59a602d3b20] to archive 2023-02-13 16:54:30,469 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:30,475 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,477 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,478 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,480 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,482 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-73fbfcf85bf94e959d070ec78e3f0f13 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-73fbfcf85bf94e959d070ec78e3f0f13 2023-02-13 16:54:30,484 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-cf1ed060c9844ab997f8b7cc03c90911 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-cf1ed060c9844ab997f8b7cc03c90911 2023-02-13 16:54:30,486 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-714a8f4733484303babdf59a602d3b20 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-714a8f4733484303babdf59a602d3b20 2023-02-13 16:54:30,507 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-41edc9dcef45425d89f150b5cec3f5ca, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-5b2d72ee795a47c780dc300dd8a1dcb2, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-4376f3f7c56647edabc6bc093e2453d2] to archive 2023-02-13 16:54:30,508 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:30,510 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,512 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,513 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,514 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,516 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-41edc9dcef45425d89f150b5cec3f5ca to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-41edc9dcef45425d89f150b5cec3f5ca 2023-02-13 16:54:30,517 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-5b2d72ee795a47c780dc300dd8a1dcb2 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-5b2d72ee795a47c780dc300dd8a1dcb2 2023-02-13 16:54:30,519 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-4376f3f7c56647edabc6bc093e2453d2 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/testSplitTableRegionUnevenDaughter=13c13f79b91b6c01ec9b3edbb20ead87-4376f3f7c56647edabc6bc093e2453d2 2023-02-13 16:54:30,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:54:30,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:30,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73. 2023-02-13 16:54:30,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8b86be70872f3798a9e95e2f1e613a73: Waiting for close lock at 1676307270455Running coprocessor pre-close hooks at 1676307270455Disabling compacts and flushes for region at 1676307270455Disabling writes for close at 1676307270455Writing region close event to WAL at 1676307270520 (+65 ms)Running coprocessor post-close hooks at 1676307270524 (+4 ms)Closed at 1676307270525 (+1 ms) 2023-02-13 16:54:30,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:30,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:30,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing fea250aff2133b2f109bfd4e8765243e, disabling compactions & flushes 2023-02-13 16:54:30,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:30,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:30,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. after waiting 0 ms 2023-02-13 16:54:30,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:30,527 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=8b86be70872f3798a9e95e2f1e613a73, regionState=CLOSED 2023-02-13 16:54:30,528 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307270527"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307270527"}]},"ts":"1676307270527"} 2023-02-13 16:54:30,528 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5-bottom] to archive 2023-02-13 16:54:30,529 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:30,531 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/317314500b084b0fbc0843178fd8dfa1.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-13 16:54:30,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 8b86be70872f3798a9e95e2f1e613a73, server=jenkins-hbase4.apache.org,38051,1676307263041 in 232 msec 2023-02-13 16:54:30,533 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=8b86be70872f3798a9e95e2f1e613a73, UNASSIGN in 239 msec 2023-02-13 16:54:30,535 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/04feb39ef4524fff89bce04d2ff24411.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,536 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/1f8a17c22d07426ea4b90d96b99bbee5.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,538 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039-bottom] to archive 2023-02-13 16:54:30,538 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:30,540 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/a0669c81f92f4316b8f40da3b6c33dbd.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,541 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/65aae0558ae048a5b5475aea1f66b32d.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,542 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/6271c1b16ce7414ea073676efb8b484e.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,543 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/07e02ed32994456cb72efcfb4a36d039.13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:54:30,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:30,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e. 2023-02-13 16:54:30,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for fea250aff2133b2f109bfd4e8765243e: Waiting for close lock at 1676307270527Running coprocessor pre-close hooks at 1676307270527Disabling compacts and flushes for region at 1676307270527Disabling writes for close at 1676307270527Writing region close event to WAL at 1676307270545 (+18 ms)Running coprocessor post-close hooks at 1676307270548 (+3 ms)Closed at 1676307270549 (+1 ms) 2023-02-13 16:54:30,551 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:30,552 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fea250aff2133b2f109bfd4e8765243e, regionState=CLOSED 2023-02-13 16:54:30,552 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307270552"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307270552"}]},"ts":"1676307270552"} 2023-02-13 16:54:30,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-13 16:54:30,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure fea250aff2133b2f109bfd4e8765243e, server=jenkins-hbase4.apache.org,38051,1676307263041 in 244 msec 2023-02-13 16:54:30,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-13 16:54:30,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fea250aff2133b2f109bfd4e8765243e, UNASSIGN in 264 msec 2023-02-13 16:54:30,560 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307270560"}]},"ts":"1676307270560"} 2023-02-13 16:54:30,562 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-13 16:54:30,565 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-13 16:54:30,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 287 msec 2023-02-13 16:54:30,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 16:54:30,602 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-13 16:54:30,607 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,615 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,617 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 16:54:30,624 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,624 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:30,624 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:30,628 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/recovered.edits] 2023-02-13 16:54:30,628 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/recovered.edits] 2023-02-13 16:54:30,628 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/recovered.edits] 2023-02-13 16:54:30,632 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/2d1742f6102f44f284b2694b0f51ad06 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf1/2d1742f6102f44f284b2694b0f51ad06 2023-02-13 16:54:30,632 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/99eca88161864a1f92b5742e974c7fa1 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf1/99eca88161864a1f92b5742e974c7fa1 2023-02-13 16:54:30,632 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/04feb39ef4524fff89bce04d2ff24411 2023-02-13 16:54:30,634 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/1f8a17c22d07426ea4b90d96b99bbee5 2023-02-13 16:54:30,636 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/a7358a66fa4c41ccb630e1072c595158 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/cf2/a7358a66fa4c41ccb630e1072c595158 2023-02-13 16:54:30,636 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/194feedc4ba34676aed7bc60ef7237b2 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/cf2/194feedc4ba34676aed7bc60ef7237b2 2023-02-13 16:54:30,636 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/317314500b084b0fbc0843178fd8dfa1 2023-02-13 16:54:30,640 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/714a8f4733484303babdf59a602d3b20 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/714a8f4733484303babdf59a602d3b20 2023-02-13 16:54:30,640 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e/recovered.edits/90.seqid 2023-02-13 16:54:30,640 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73/recovered.edits/90.seqid 2023-02-13 16:54:30,642 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/fea250aff2133b2f109bfd4e8765243e 2023-02-13 16:54:30,642 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:30,642 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/73fbfcf85bf94e959d070ec78e3f0f13 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/73fbfcf85bf94e959d070ec78e3f0f13 2023-02-13 16:54:30,644 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/952f0a7b5d974f4bb559f0ff9ba2ce4d 2023-02-13 16:54:30,645 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/cf1ed060c9844ab997f8b7cc03c90911 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf1/cf1ed060c9844ab997f8b7cc03c90911 2023-02-13 16:54:30,648 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/07e02ed32994456cb72efcfb4a36d039 2023-02-13 16:54:30,650 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/41edc9dcef45425d89f150b5cec3f5ca to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/41edc9dcef45425d89f150b5cec3f5ca 2023-02-13 16:54:30,652 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/4376f3f7c56647edabc6bc093e2453d2 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/4376f3f7c56647edabc6bc093e2453d2 2023-02-13 16:54:30,653 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/5b2d72ee795a47c780dc300dd8a1dcb2 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/5b2d72ee795a47c780dc300dd8a1dcb2 2023-02-13 16:54:30,655 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/6271c1b16ce7414ea073676efb8b484e 2023-02-13 16:54:30,656 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/65aae0558ae048a5b5475aea1f66b32d 2023-02-13 16:54:30,658 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/cf2/a0669c81f92f4316b8f40da3b6c33dbd 2023-02-13 16:54:30,662 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/recovered.edits/85.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87/recovered.edits/85.seqid 2023-02-13 16:54:30,662 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionUnevenDaughter/13c13f79b91b6c01ec9b3edbb20ead87 2023-02-13 16:54:30,662 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 16:54:30,666 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,688 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-13 16:54:30,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 16:54:30,731 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-13 16:54:30,732 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,732 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-13 16:54:30,733 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307270733"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:30,733 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307270733"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:30,733 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307270733"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:30,736 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 16:54:30,736 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 13c13f79b91b6c01ec9b3edbb20ead87, NAME => 'testSplitTableRegionUnevenDaughter,,1676307265803.13c13f79b91b6c01ec9b3edbb20ead87.', STARTKEY => '', ENDKEY => ''}, {ENCODED => fea250aff2133b2f109bfd4e8765243e, NAME => 'testSplitTableRegionUnevenDaughter,,1676307268316.fea250aff2133b2f109bfd4e8765243e.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 8b86be70872f3798a9e95e2f1e613a73, NAME => 'testSplitTableRegionUnevenDaughter,26,1676307268316.8b86be70872f3798a9e95e2f1e613a73.', STARTKEY => '26', ENDKEY => ''}] 2023-02-13 16:54:30,736 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-13 16:54:30,736 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307270736"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:30,738 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-13 16:54:30,741 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:30,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 133 msec 2023-02-13 16:54:30,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 16:54:30,921 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-13 16:54:30,938 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=392 (was 360) Potentially hanging thread: hconnection-0x26bb8ba6-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:55566 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26bb8ba6-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26bb8ba6-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:54114 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:52702 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:55576 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:53958 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:52736 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26bb8ba6-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:54112 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:55292 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:52414 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:38051-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:55346 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:52764 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:52718 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:52722 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:52794 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:52384 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-572501980_15 at /127.0.0.1:53804 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=841 (was 700) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=260 (was 283), ProcessCount=168 (was 168), AvailableMemoryMB=1538 (was 1634) 2023-02-13 16:54:30,953 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=392, OpenFileDescriptor=841, MaxFileDescriptor=60000, SystemLoadAverage=260, ProcessCount=168, AvailableMemoryMB=1537 2023-02-13 16:54:30,953 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:30,958 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:54:30,959 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:54:30,960 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:30,962 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:30,963 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:30,964 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:30,965 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea empty. 2023-02-13 16:54:30,966 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:30,966 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 16:54:30,977 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:30,978 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7c340e1b244f9866f9e7640b1e7f78ea, NAME => 'testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:54:30,989 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:30,990 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 7c340e1b244f9866f9e7640b1e7f78ea, disabling compactions & flushes 2023-02-13 16:54:30,990 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:30,990 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:30,990 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. after waiting 0 ms 2023-02-13 16:54:30,990 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:30,990 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:30,990 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 7c340e1b244f9866f9e7640b1e7f78ea: Waiting for close lock at 1676307270990Disabling compacts and flushes for region at 1676307270990Disabling writes for close at 1676307270990Writing region close event to WAL at 1676307270990Closed at 1676307270990 2023-02-13 16:54:30,992 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:30,993 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307270993"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307270993"}]},"ts":"1676307270993"} 2023-02-13 16:54:30,996 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:30,997 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:30,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307270997"}]},"ts":"1676307270997"} 2023-02-13 16:54:30,998 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-13 16:54:31,001 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:54:31,003 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:54:31,003 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:54:31,003 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:54:31,003 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:54:31,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, ASSIGN}] 2023-02-13 16:54:31,005 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, ASSIGN 2023-02-13 16:54:31,007 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:31,157 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:54:31,158 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7c340e1b244f9866f9e7640b1e7f78ea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,158 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307271158"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307271158"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307271158"}]},"ts":"1676307271158"} 2023-02-13 16:54:31,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:31,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:31,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7c340e1b244f9866f9e7640b1e7f78ea, NAME => 'testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:31,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:31,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:31,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:31,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:31,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:31,318 INFO [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:31,320 DEBUG [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1 2023-02-13 16:54:31,320 DEBUG [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1 2023-02-13 16:54:31,320 INFO [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c340e1b244f9866f9e7640b1e7f78ea columnFamilyName cf1 2023-02-13 16:54:31,321 INFO [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] regionserver.HStore(310): Store=7c340e1b244f9866f9e7640b1e7f78ea/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:31,321 INFO [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:31,322 DEBUG [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2 2023-02-13 16:54:31,322 DEBUG [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2 2023-02-13 16:54:31,324 INFO [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c340e1b244f9866f9e7640b1e7f78ea columnFamilyName cf2 2023-02-13 16:54:31,324 INFO [StoreOpener-7c340e1b244f9866f9e7640b1e7f78ea-1] regionserver.HStore(310): Store=7c340e1b244f9866f9e7640b1e7f78ea/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:31,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:31,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:31,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:31,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:31,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:31,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7c340e1b244f9866f9e7640b1e7f78ea; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71744081, jitterRate=0.06907011568546295}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:31,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7c340e1b244f9866f9e7640b1e7f78ea: Running coprocessor pre-open hook at 1676307271317Writing region info on filesystem at 1676307271317Initializing all the Stores at 1676307271318 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307271318Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307271318Cleaning up temporary data from old regions at 1676307271326 (+8 ms)Cleaning up detritus from prior splits at 1676307271327 (+1 ms)Running coprocessor post-open hooks at 1676307271331 (+4 ms)Region opened successfully at 1676307271331 2023-02-13 16:54:31,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea., pid=27, masterSystemTime=1676307271312 2023-02-13 16:54:31,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:31,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:31,335 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7c340e1b244f9866f9e7640b1e7f78ea, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,335 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307271335"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307271335"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307271335"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307271335"}]},"ts":"1676307271335"} 2023-02-13 16:54:31,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-13 16:54:31,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 in 177 msec 2023-02-13 16:54:31,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-13 16:54:31,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, ASSIGN in 337 msec 2023-02-13 16:54:31,343 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:31,343 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307271343"}]},"ts":"1676307271343"} 2023-02-13 16:54:31,344 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-13 16:54:31,352 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:31,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 393 msec 2023-02-13 16:54:31,471 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:31,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 16:54:31,473 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 16:54:31,474 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,474 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 16:54:31,474 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 16:54:31,475 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,475 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,476 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,476 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,476 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,476 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,477 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,477 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:31,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:31,478 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:31,478 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,478 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:31,479 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:31,479 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:31,479 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:31,479 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,479 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:31,480 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:31,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:31,480 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:31,480 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:31,480 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:31,480 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,481 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:31,481 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:31,481 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:31,481 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:31,481 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. started... 2023-02-13 16:54:31,482 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c340e1b244f9866f9e7640b1e7f78ea 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 16:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,483 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,483 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,483 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,484 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,484 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,493 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/0d036ec9f8b14c7aa964d7fa52faf3de 2023-02-13 16:54:31,509 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/1fac80fd9837409e9bf92f6acedb3be4 2023-02-13 16:54:31,514 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/0d036ec9f8b14c7aa964d7fa52faf3de as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de 2023-02-13 16:54:31,519 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:54:31,520 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/1fac80fd9837409e9bf92f6acedb3be4 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4 2023-02-13 16:54:31,525 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:54:31,526 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7c340e1b244f9866f9e7640b1e7f78ea in 44ms, sequenceid=6, compaction requested=false 2023-02-13 16:54:31,526 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-13 16:54:31,526 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c340e1b244f9866f9e7640b1e7f78ea: 2023-02-13 16:54:31,526 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:31,527 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:31,527 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:31,527 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,527 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:31,527 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:31,530 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,530 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,530 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,530 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 16:54:31,533 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 16:54:31,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@81af6f2[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:31,533 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,534 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,534 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,534 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,534 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:31,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,534 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:31,534 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,534 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:31,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,534 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:31,534 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:31,535 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:31,535 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:31,535 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:31,535 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:31,535 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:31,536 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:31,536 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,536 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:31,537 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:31,537 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,537 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,537 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,537 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,538 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:31,538 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:31,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:31,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 16:54:31,543 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 16:54:31,543 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:31,543 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:31,543 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 16:54:31,543 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:31,546 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,546 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,546 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,546 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,546 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,547 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,547 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,547 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,548 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,549 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,550 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,551 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,552 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,555 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,555 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,555 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,555 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,555 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,555 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,555 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 16:54:31,556 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,555 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,556 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:31,555 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,557 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:31,557 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:31,555 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,557 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,557 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,557 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,557 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,558 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,558 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,558 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,558 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,657 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:31,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:31,676 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:31,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:31,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 16:54:31,678 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 16:54:31,678 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,679 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 16:54:31,679 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 16:54:31,679 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,679 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,681 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,681 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,681 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,681 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,681 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,681 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:31,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:31,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:31,683 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:31,683 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:31,683 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:31,683 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,683 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:31,684 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:31,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:31,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,685 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:31,685 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:31,685 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:31,685 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:31,686 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:31,686 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:31,686 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:31,686 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. started... 2023-02-13 16:54:31,686 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c340e1b244f9866f9e7640b1e7f78ea 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:31,687 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,689 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,689 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,689 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,689 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,689 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,699 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/8efe1e3861db401591b145900ec85de4 2023-02-13 16:54:31,713 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/ea148b6afa9d4fae941a628f2a6f8ce2 2023-02-13 16:54:31,718 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/8efe1e3861db401591b145900ec85de4 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4 2023-02-13 16:54:31,723 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:54:31,724 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/ea148b6afa9d4fae941a628f2a6f8ce2 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2 2023-02-13 16:54:31,728 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:54:31,729 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7c340e1b244f9866f9e7640b1e7f78ea in 43ms, sequenceid=19, compaction requested=false 2023-02-13 16:54:31,730 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c340e1b244f9866f9e7640b1e7f78ea: 2023-02-13 16:54:31,730 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:31,730 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:31,730 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:31,730 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,730 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:31,730 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:31,732 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,732 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,732 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,732 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 16:54:31,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ce441cf[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:31,734 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 16:54:31,734 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,735 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,735 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,735 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,735 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:31,735 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,735 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,735 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:31,735 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:31,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,735 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:31,735 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:31,735 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,736 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:31,736 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:31,736 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:31,737 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:31,737 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,737 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:31,741 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:31,741 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:31,742 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,742 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:31,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:31,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 16:54:31,746 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 16:54:31,746 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:31,746 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:31,746 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 16:54:31,746 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:31,747 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,747 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,747 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,747 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,747 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,747 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,748 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,748 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,748 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,749 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,750 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,751 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,751 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,755 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,754 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,755 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,755 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,755 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,755 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,755 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 16:54:31,756 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:31,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,756 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,756 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,756 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:31,756 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:31,756 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,857 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:31,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:31,875 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:31,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:31,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 16:54:31,877 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 16:54:31,877 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,878 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 16:54:31,878 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 16:54:31,878 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,878 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,880 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,880 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,880 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,880 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,884 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,884 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:31,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,885 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:31,886 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:31,886 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:31,886 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:31,886 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:31,886 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,886 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:31,886 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:31,886 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:31,886 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:31,889 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:31,889 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:31,889 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:31,889 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:31,889 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:31,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:31,889 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:31,889 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:31,889 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,890 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:31,890 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:31,890 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:31,890 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:31,890 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. started... 2023-02-13 16:54:31,890 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c340e1b244f9866f9e7640b1e7f78ea 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:31,892 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,892 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,892 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,892 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,892 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,892 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,903 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/aa0a8f0505554105be3aa6029bdd2281 2023-02-13 16:54:31,917 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/cddceb83e7964154a2688e1751c7c3c8 2023-02-13 16:54:31,922 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/aa0a8f0505554105be3aa6029bdd2281 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281 2023-02-13 16:54:31,927 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:54:31,928 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/cddceb83e7964154a2688e1751c7c3c8 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8 2023-02-13 16:54:31,933 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:54:31,934 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7c340e1b244f9866f9e7640b1e7f78ea in 44ms, sequenceid=32, compaction requested=true 2023-02-13 16:54:31,934 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c340e1b244f9866f9e7640b1e7f78ea: 2023-02-13 16:54:31,934 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:31,934 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:31,934 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:31,934 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,934 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:31,934 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:31,935 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,935 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,936 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,936 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,938 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 16:54:31,938 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f8bbed6[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:31,938 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 16:54:31,938 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,939 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,939 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,939 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,939 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,939 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:31,940 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:31,940 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:31,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,940 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:31,940 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:31,940 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:31,940 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:31,940 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:31,940 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:31,940 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:31,941 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:31,941 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,941 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:31,943 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:31,943 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:31,943 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,943 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,943 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,944 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:31,944 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:31,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:31,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 16:54:31,948 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 16:54:31,948 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:31,948 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:31,948 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 16:54:31,948 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:31,950 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,950 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,950 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,950 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,950 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,950 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,950 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,951 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,951 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,951 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,953 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:31,953 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,954 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,958 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:31,958 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:31,958 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 16:54:31,958 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,958 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,957 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,958 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:31,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,958 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:31,958 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,958 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:31,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,958 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:31,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,994 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 16:54:31,995 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 16:54:31,995 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:31,996 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-13 16:54:32,059 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:32,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:32,079 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:32,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:32,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 16:54:32,081 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 16:54:32,081 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,082 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 16:54:32,082 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 16:54:32,082 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,082 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,083 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,084 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,084 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,084 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,084 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,084 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:32,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:32,085 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:32,085 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:32,085 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:32,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:32,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:32,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:32,087 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,087 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:32,088 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:32,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:32,088 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:32,088 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:32,088 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:32,088 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,088 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,089 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:32,090 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:32,090 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:32,090 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:32,090 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:32,090 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:32,090 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:32,090 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. started... 2023-02-13 16:54:32,091 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c340e1b244f9866f9e7640b1e7f78ea 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:32,098 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,099 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,099 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,105 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/a02e61f2ddb14cacafccac50add8ac4c 2023-02-13 16:54:32,118 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/d2842e91777c42b1a9826f67c663896e 2023-02-13 16:54:32,123 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/a02e61f2ddb14cacafccac50add8ac4c as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c 2023-02-13 16:54:32,127 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:54:32,128 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/d2842e91777c42b1a9826f67c663896e as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e 2023-02-13 16:54:32,133 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:54:32,134 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7c340e1b244f9866f9e7640b1e7f78ea in 43ms, sequenceid=45, compaction requested=true 2023-02-13 16:54:32,134 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c340e1b244f9866f9e7640b1e7f78ea: 2023-02-13 16:54:32,134 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:32,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:32,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:32,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:32,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:32,136 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,136 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,136 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,136 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 16:54:32,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a26ffc1[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:32,138 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 16:54:32,138 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,140 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:32,140 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:32,140 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:32,140 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:32,140 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:32,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,140 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,140 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:32,140 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:32,140 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:32,140 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:32,140 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:32,142 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,142 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:32,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,142 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,143 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:32,144 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:32,144 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,144 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:32,145 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:32,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,145 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,145 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:32,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:32,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 16:54:32,149 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 16:54:32,149 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:32,149 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:32,149 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 16:54:32,149 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:32,150 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,150 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,150 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,150 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,151 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,150 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,152 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,152 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,154 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,154 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,154 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,158 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,158 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,158 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,158 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,158 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,158 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,158 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,158 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,159 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,159 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,159 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,159 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,159 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,159 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,159 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,159 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,159 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,160 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 16:54:32,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:32,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,161 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:32,161 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:32,261 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:32,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:32,282 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:32,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:32,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 16:54:32,284 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 16:54:32,284 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,284 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 16:54:32,284 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 16:54:32,285 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,285 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,286 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,286 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,286 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,286 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,287 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,287 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:32,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,288 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:32,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,288 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:32,289 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:32,289 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:32,289 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:32,290 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:32,290 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. started... 2023-02-13 16:54:32,290 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c340e1b244f9866f9e7640b1e7f78ea 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:32,291 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,292 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,292 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,292 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,292 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,292 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,305 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/b9b4c73c64b84b5aadda40c165ac3307 2023-02-13 16:54:32,323 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/fd29c58bdc044b5693e1d460d1b4d8bf 2023-02-13 16:54:32,330 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/b9b4c73c64b84b5aadda40c165ac3307 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307 2023-02-13 16:54:32,335 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:54:32,336 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/fd29c58bdc044b5693e1d460d1b4d8bf as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf 2023-02-13 16:54:32,340 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:54:32,342 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7c340e1b244f9866f9e7640b1e7f78ea in 52ms, sequenceid=58, compaction requested=true 2023-02-13 16:54:32,342 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c340e1b244f9866f9e7640b1e7f78ea: 2023-02-13 16:54:32,342 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:32,342 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:32,342 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:32,342 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,342 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:32,342 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:32,345 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,345 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,345 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,345 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 16:54:32,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55d43511[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:32,347 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 16:54:32,348 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:32,349 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:32,349 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:32,349 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:32,349 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:32,349 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:32,349 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:32,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,349 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:32,349 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:32,350 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:32,351 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,351 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:32,351 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,351 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:32,352 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:32,352 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,352 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:32,353 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:32,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,353 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,353 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:32,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:32,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 16:54:32,356 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 16:54:32,356 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:32,356 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:32,356 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 16:54:32,356 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:32,358 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,358 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,358 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,358 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,359 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,359 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,361 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,361 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,362 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,367 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,367 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,367 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,367 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,367 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,367 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,367 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 16:54:32,368 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:32,367 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,368 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:32,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,369 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,369 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,369 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,469 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:32,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:32,487 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:32,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:32,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 16:54:32,489 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 16:54:32,489 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,489 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 16:54:32,489 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 16:54:32,489 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,489 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,492 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,492 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,492 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,492 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:32,493 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:32,493 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 16:54:32,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:32,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:32,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:32,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:32,494 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:32,494 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. started... 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:32,495 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c340e1b244f9866f9e7640b1e7f78ea 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:32,496 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,496 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,497 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,497 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,497 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,497 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,509 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/1eeb75e7149d43e2badffd59df6be7d5 2023-02-13 16:54:32,535 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/ec1932c9a6a549fc869dd2cda2526870 2023-02-13 16:54:32,544 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/1eeb75e7149d43e2badffd59df6be7d5 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5 2023-02-13 16:54:32,553 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:54:32,554 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/ec1932c9a6a549fc869dd2cda2526870 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870 2023-02-13 16:54:32,560 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:54:32,561 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7c340e1b244f9866f9e7640b1e7f78ea in 67ms, sequenceid=71, compaction requested=true 2023-02-13 16:54:32,562 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c340e1b244f9866f9e7640b1e7f78ea: 2023-02-13 16:54:32,562 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:32,562 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:32,562 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:32,562 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,562 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 16:54:32,562 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 16:54:32,563 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,564 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,564 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,564 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 16:54:32,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@be0ec5d[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:32,567 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 16:54:32,567 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,573 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,573 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,573 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,573 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:32,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,573 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,574 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,574 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:32,574 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 16:54:32,574 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:32,573 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:32,574 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:32,574 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 16:54:32,574 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:32,574 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:32,575 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:32,575 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:32,575 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,575 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,575 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:32,575 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:32,576 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,576 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 16:54:32,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,576 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,576 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 16:54:32,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:32,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 16:54:32,579 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 16:54:32,579 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:32,579 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:32,579 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 16:54:32,579 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:32,581 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,581 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,582 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,583 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,583 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,585 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 16:54:32,585 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,585 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,589 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,589 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,589 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,589 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,589 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,589 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,589 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,589 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 16:54:32,590 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:32,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,589 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,590 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,590 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:32,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,590 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,590 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:32,591 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,591 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 16:54:32,591 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,591 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,591 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:32,591 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:32,591 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 16:54:32,591 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 16:54:32,691 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:32,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:32,955 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 16:54:32,956 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:32,958 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:32,959 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:32,959 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:32,959 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:32,960 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:32,960 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:32,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:32,961 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:33,208 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,209 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:33,210 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:33,210 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:33,212 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-13 16:54:33,213 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:33,213 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:33,213 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:33,213 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:33,213 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:33,217 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 16:54:33,218 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:33,219 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:33,219 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:33,219 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:33,220 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:33,220 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:33,220 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:33,221 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:33,221 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:33,222 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,222 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:33,222 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,222 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,222 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,222 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 16:54:33,223 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:33,224 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:33,224 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307303224 2023-02-13 16:54:33,226 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:33,226 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,230 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:33,231 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:33,231 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:33,231 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307333231 2023-02-13 16:54:33,231 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307393231 2023-02-13 16:54:33,231 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 16:54:33,232 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:33,234 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:33,234 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:33,234 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:33,482 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,482 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:33,482 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:33,482 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:33,482 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:33,482 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:33,482 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:33,482 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:33,483 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:33,483 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:33,485 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 16:54:33,485 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:33,486 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:33,486 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:33,486 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:33,487 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:33,487 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:33,487 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:33,487 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:33,488 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:33,488 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,488 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:33,488 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,488 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,488 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,489 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 16:54:33,489 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:33,490 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:33,492 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307303492 2023-02-13 16:54:33,492 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:33,493 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,496 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:33,496 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:33,497 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:33,497 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307333497 2023-02-13 16:54:33,497 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307393497 2023-02-13 16:54:33,497 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:33,497 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:33,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN}] 2023-02-13 16:54:33,499 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:33,499 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:33,499 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:33,747 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,747 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:33,747 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:33,747 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:33,747 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:33,748 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:33,748 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:33,748 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:33,748 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:33,749 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:33,750 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 16:54:33,750 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:33,751 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:33,751 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:33,751 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:33,752 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:33,752 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:33,753 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:33,753 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:33,753 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:33,754 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,754 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:33,754 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,754 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,754 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,754 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 16:54:33,755 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:33,755 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:33,756 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307303756 2023-02-13 16:54:33,757 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:33,757 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:33,760 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:33,760 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:33,760 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:33,760 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307333760 2023-02-13 16:54:33,760 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307393760 2023-02-13 16:54:33,761 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:54:33,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN}] 2023-02-13 16:54:33,761 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:33,763 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:33,764 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:33,764 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307273764"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307273764"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307273764"}]},"ts":"1676307273764"} 2023-02-13 16:54:33,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:33,766 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:33,766 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:33,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:33,766 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:34,011 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,011 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:34,011 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:34,011 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:34,011 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:34,011 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:34,011 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:34,012 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:34,012 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:34,012 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:34,014 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:54:34,014 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 16:54:34,014 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:34,014 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:34,015 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:34,015 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:34,015 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:34,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:34,017 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,018 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,018 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:34,018 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,018 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,018 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,018 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:34,018 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,018 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 16:54:34,019 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:34,019 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:34,019 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea to restore RIT 2023-02-13 16:54:34,020 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307304020 2023-02-13 16:54:34,021 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,021 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:34,023 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:34,023 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:34,024 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:34,024 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307334024 2023-02-13 16:54:34,024 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307394024 2023-02-13 16:54:34,024 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:54:34,024 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:34,024 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307274024"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307274024"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307274024"}]},"ts":"1676307274024"} 2023-02-13 16:54:34,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:34,026 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:34,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7c340e1b244f9866f9e7640b1e7f78ea, disabling compactions & flushes 2023-02-13 16:54:34,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:34,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:34,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. after waiting 0 ms 2023-02-13 16:54:34,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:34,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 7c340e1b244f9866f9e7640b1e7f78ea 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 16:54:34,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/85b851389d8d4e5e9a20a801a85d064b 2023-02-13 16:54:34,207 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/1dde9df079304e60a4f8753ae72a4c43 2023-02-13 16:54:34,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf1/85b851389d8d4e5e9a20a801a85d064b as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b 2023-02-13 16:54:34,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:54:34,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.tmp/cf2/1dde9df079304e60a4f8753ae72a4c43 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43 2023-02-13 16:54:34,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:54:34,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7c340e1b244f9866f9e7640b1e7f78ea in 44ms, sequenceid=82, compaction requested=true 2023-02-13 16:54:34,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 16:54:34,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:34,244 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. 2023-02-13 16:54:34,245 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7c340e1b244f9866f9e7640b1e7f78ea: Waiting for close lock at 1676307274180Running coprocessor pre-close hooks at 1676307274180Disabling compacts and flushes for region at 1676307274180Disabling writes for close at 1676307274180Obtaining lock to block concurrent updates at 1676307274180Preparing flush snapshotting stores in 7c340e1b244f9866f9e7640b1e7f78ea at 1676307274180Finished memstore snapshotting testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676307274180Flushing stores of testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. at 1676307274181 (+1 ms)Flushing 7c340e1b244f9866f9e7640b1e7f78ea/cf1: creating writer at 1676307274182 (+1 ms)Flushing 7c340e1b244f9866f9e7640b1e7f78ea/cf1: appending metadata at 1676307274184 (+2 ms)Flushing 7c340e1b244f9866f9e7640b1e7f78ea/cf1: closing flushed file at 1676307274184Flushing 7c340e1b244f9866f9e7640b1e7f78ea/cf2: creating writer at 1676307274198 (+14 ms)Flushing 7c340e1b244f9866f9e7640b1e7f78ea/cf2: appending metadata at 1676307274200 (+2 ms)Flushing 7c340e1b244f9866f9e7640b1e7f78ea/cf2: closing flushed file at 1676307274200Flushing 7c340e1b244f9866f9e7640b1e7f78ea/cf1: reopening flushed file at 1676307274213 (+13 ms)Flushing 7c340e1b244f9866f9e7640b1e7f78ea/cf2: reopening flushed file at 1676307274219 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7c340e1b244f9866f9e7640b1e7f78ea in 44ms, sequenceid=82, compaction requested=true at 1676307274224 (+5 ms)Writing region close event to WAL at 1676307274240 (+16 ms)Running coprocessor post-close hooks at 1676307274244 (+4 ms)Closed at 1676307274244 2023-02-13 16:54:34,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,248 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSED 2023-02-13 16:54:34,248 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307274247"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307274247"}]},"ts":"1676307274247"} 2023-02-13 16:54:34,249 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:34,249 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:34,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:34,250 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:34,274 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,274 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:34,274 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:34,274 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:34,275 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:34,275 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:34,275 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:34,275 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:34,275 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:34,276 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:34,277 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:54:34,278 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 16:54:34,278 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 16:54:34,278 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:34,279 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:34,279 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:34,279 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:34,280 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,280 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:34,281 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,281 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,281 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:34,282 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,282 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,282 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:34,283 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:34,283 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,283 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,283 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,283 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:34,283 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,283 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:34,283 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 16:54:34,284 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:34,285 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:34,285 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea to restore RIT 2023-02-13 16:54:34,285 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307304285 2023-02-13 16:54:34,285 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:34,286 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:34,288 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:34,289 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:34,289 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:34,289 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307334289 2023-02-13 16:54:34,289 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307394289 2023-02-13 16:54:34,289 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:54:34,289 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:34,289 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307274289"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307274289"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307274289"}]},"ts":"1676307274289"} 2023-02-13 16:54:34,291 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:34,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-13 16:54:34,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 in 265 msec 2023-02-13 16:54:34,294 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:34,294 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:34,295 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:34,539 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,539 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:34,539 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:34,540 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:34,540 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:34,540 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:34,540 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:34,540 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:34,540 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:34,541 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:34,542 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:54:34,542 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 16:54:34,542 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:34,542 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:34,543 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:34,543 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:34,544 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:34,544 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,544 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:34,545 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,545 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,545 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:34,546 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,546 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,546 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:34,546 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:34,546 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,546 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,546 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,546 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:34,546 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:34,546 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 16:54:34,547 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:34,547 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:34,548 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea to restore RIT 2023-02-13 16:54:34,548 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307304548 2023-02-13 16:54:34,549 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:34,549 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,551 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:34,551 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:34,552 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:34,552 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307334552 2023-02-13 16:54:34,552 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307394552 2023-02-13 16:54:34,552 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:54:34,552 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,554 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:34,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-13 16:54:34,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN in 793 msec 2023-02-13 16:54:34,560 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:34,560 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:34,560 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:34,802 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,802 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:34,802 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:34,802 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:34,803 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:34,803 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:34,803 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:34,803 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:34,803 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:34,804 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:34,805 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:54:34,805 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:34,805 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:34,806 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:34,806 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:34,807 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:34,807 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:34,807 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,808 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:34,808 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,809 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:34,809 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:34,810 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,810 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:34,810 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:34,810 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:34,810 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:34,810 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:34,810 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 16:54:34,811 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:34,811 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:34,811 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307304811 2023-02-13 16:54:34,812 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:34,813 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,815 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:34,816 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:34,816 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:34,816 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307334816 2023-02-13 16:54:34,816 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307394816 2023-02-13 16:54:34,816 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:34,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:34,821 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:34,831 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=7c340e1b244f9866f9e7640b1e7f78ea, threads=14 2023-02-13 16:54:34,832 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,832 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,832 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,832 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,832 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,833 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,833 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,833 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,833 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,833 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,834 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,834 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,834 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,835 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,960 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,968 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,977 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,984 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,986 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,997 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,998 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:34,999 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,004 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,005 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,005 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,007 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,007 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,012 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,012 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 7c340e1b244f9866f9e7640b1e7f78ea Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 16:54:35,056 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:35,056 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:35,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:35,057 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:35,067 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:35,067 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:35,067 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:35,067 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:35,068 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:35,068 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:35,068 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:35,068 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:35,068 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:35,069 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:35,070 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 16:54:35,070 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:35,070 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:35,071 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:35,071 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:35,071 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:35,072 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:35,072 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:35,072 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:35,073 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:35,073 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:35,074 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:35,074 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:35,074 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:35,075 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:35,075 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:35,075 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:35,075 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:35,075 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:35,075 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:35,075 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:35,075 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 16:54:35,076 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:35,076 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:35,077 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307305077 2023-02-13 16:54:35,078 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:35,078 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:35,080 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:35,081 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:35,081 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:35,081 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307335081 2023-02-13 16:54:35,081 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307395081 2023-02-13 16:54:35,081 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:54:35,090 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=7c340e1b244f9866f9e7640b1e7f78ea, threads=14 2023-02-13 16:54:35,091 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,091 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,091 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,091 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,091 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,092 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,092 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,092 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,092 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,092 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,092 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,096 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,096 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,096 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,201 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:35,337 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,339 WARN [DataStreamer for file /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/.splits/4cc157038f0cfa34e35dda26617b6793/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea] hdfs.DataStreamer(982): Caught exception java.lang.InterruptedException at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1257) at java.lang.Thread.join(Thread.java:1331) at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980) at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:630) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:807) 2023-02-13 16:54:35,340 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,340 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,356 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,376 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,376 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,376 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,386 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,396 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,396 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,397 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,400 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,402 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,801 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870 for region: 7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:35,801 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 7c340e1b244f9866f9e7640b1e7f78ea Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 16:54:35,842 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:35,847 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:54:35,849 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:54:35,849 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:35,849 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:35,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:35,849 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:36,081 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,082 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:36,082 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:36,082 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:36,082 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:36,082 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:36,082 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:36,082 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:36,082 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:36,083 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:36,084 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 16:54:36,084 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:36,084 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:36,085 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:36,085 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:36,086 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:36,086 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:36,086 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:36,087 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:36,087 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:36,087 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:36,088 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:36,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:36,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:36,089 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:36,089 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,089 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,089 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,089 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:36,090 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:36,090 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 16:54:36,090 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:36,091 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:36,091 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307306091 2023-02-13 16:54:36,091 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:36,093 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,095 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:36,095 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:36,096 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:36,096 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307336096 2023-02-13 16:54:36,096 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307396096 2023-02-13 16:54:36,096 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:36,098 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:36,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:36,100 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:36,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:36,346 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:36,346 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:36,346 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:36,346 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:36,346 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:36,347 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:36,347 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:36,347 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:36,347 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:36,347 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:36,348 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 16:54:36,349 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:36,349 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:36,349 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:36,349 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:36,349 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:36,350 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:36,350 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:36,350 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:36,350 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:36,350 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:36,351 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:36,351 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,351 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN 2023-02-13 16:54:36,351 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:36,351 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:36,352 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,352 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,352 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,352 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7c340e1b244f9866f9e7640b1e7f78ea, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:36,352 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7c340e1b244f9866f9e7640b1e7f78ea, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:36,352 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:36,352 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:36,353 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:36,353 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307306353 2023-02-13 16:54:36,355 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:36,355 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,357 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:36,357 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=7c340e1b244f9866f9e7640b1e7f78ea, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:36,357 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:36,357 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307336357 2023-02-13 16:54:36,357 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307396357 2023-02-13 16:54:36,357 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:54:36,359 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307276359"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676307276359"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676307276359"}]},"ts":"1676307276359"} 2023-02-13 16:54:36,359 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307276359"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307276359"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307276359"}]},"ts":"1676307276359"} 2023-02-13 16:54:36,359 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307276359"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307276359"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307276359"}]},"ts":"1676307276359"} 2023-02-13 16:54:36,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=11ec0bb6bbbf4d006fe9a7fb93bd9046, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4cc157038f0cfa34e35dda26617b6793, ASSIGN}] 2023-02-13 16:54:36,366 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4cc157038f0cfa34e35dda26617b6793, ASSIGN 2023-02-13 16:54:36,367 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4cc157038f0cfa34e35dda26617b6793, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:36,368 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=11ec0bb6bbbf4d006fe9a7fb93bd9046, ASSIGN 2023-02-13 16:54:36,368 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=11ec0bb6bbbf4d006fe9a7fb93bd9046, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:36,517 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 16:54:36,519 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=4cc157038f0cfa34e35dda26617b6793, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:36,519 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307276519"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307276519"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307276519"}]},"ts":"1676307276519"} 2023-02-13 16:54:36,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 4cc157038f0cfa34e35dda26617b6793, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:36,524 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=11ec0bb6bbbf4d006fe9a7fb93bd9046, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:36,524 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307276524"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307276524"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307276524"}]},"ts":"1676307276524"} 2023-02-13 16:54:36,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 11ec0bb6bbbf4d006fe9a7fb93bd9046, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:36,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:36,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 11ec0bb6bbbf4d006fe9a7fb93bd9046, NAME => 'testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 16:54:36,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:36,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:36,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:36,678 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:36,678 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:36,679 INFO [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:36,680 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1 2023-02-13 16:54:36,680 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1 2023-02-13 16:54:36,680 INFO [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11ec0bb6bbbf4d006fe9a7fb93bd9046 columnFamilyName cf1 2023-02-13 16:54:36,695 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de-bottom 2023-02-13 16:54:36,700 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5-bottom 2023-02-13 16:54:36,705 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b-bottom 2023-02-13 16:54:36,710 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4-bottom 2023-02-13 16:54:36,715 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c-bottom 2023-02-13 16:54:36,720 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281-bottom 2023-02-13 16:54:36,724 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307-bottom 2023-02-13 16:54:36,724 INFO [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(310): Store=11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:36,724 INFO [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:36,725 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2 2023-02-13 16:54:36,725 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2 2023-02-13 16:54:36,725 INFO [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11ec0bb6bbbf4d006fe9a7fb93bd9046 columnFamilyName cf2 2023-02-13 16:54:36,738 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43-bottom 2023-02-13 16:54:36,743 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4-bottom 2023-02-13 16:54:36,748 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8-bottom 2023-02-13 16:54:36,753 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e-bottom 2023-02-13 16:54:36,757 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2-bottom 2023-02-13 16:54:36,762 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870-bottom 2023-02-13 16:54:36,767 DEBUG [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf-bottom 2023-02-13 16:54:36,767 INFO [StoreOpener-11ec0bb6bbbf4d006fe9a7fb93bd9046-1] regionserver.HStore(310): Store=11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:36,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:36,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:36,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:36,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:36,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 11ec0bb6bbbf4d006fe9a7fb93bd9046; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69655874, jitterRate=0.03795340657234192}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:36,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 11ec0bb6bbbf4d006fe9a7fb93bd9046: Running coprocessor pre-open hook at 1676307276678Writing region info on filesystem at 1676307276678Initializing all the Stores at 1676307276678Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307276678Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307276679 (+1 ms)Cleaning up temporary data from old regions at 1676307276770 (+91 ms)Cleaning up detritus from prior splits at 1676307276770Running coprocessor post-open hooks at 1676307276773 (+3 ms)Region opened successfully at 1676307276773 2023-02-13 16:54:36,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046., pid=34, masterSystemTime=1676307276673 2023-02-13 16:54:36,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:36,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:54:36,776 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:36,776 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:36,776 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:36,776 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1912): 11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1 is initiating minor compaction (all files) 2023-02-13 16:54:36,776 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:36,776 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1 in testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:36,776 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1912): 11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2 is initiating minor compaction (all files) 2023-02-13 16:54:36,776 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2 in testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:36,776 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/.tmp, totalSize=34.9 K 2023-02-13 16:54:36,777 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/.tmp, totalSize=34.9 K 2023-02-13 16:54:36,777 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307271464 2023-02-13 16:54:36,777 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307271464 2023-02-13 16:54:36,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:36,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:36,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:36,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4cc157038f0cfa34e35dda26617b6793, NAME => 'testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 16:54:36,778 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307271659 2023-02-13 16:54:36,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:36,779 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=11ec0bb6bbbf4d006fe9a7fb93bd9046, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:36,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,779 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307271859 2023-02-13 16:54:36,779 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307276778"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307276778"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307276778"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307276778"}]},"ts":"1676307276778"} 2023-02-13 16:54:36,779 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307271659 2023-02-13 16:54:36,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:36,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,779 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307272062 2023-02-13 16:54:36,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,780 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307271859 2023-02-13 16:54:36,780 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307272264 2023-02-13 16:54:36,780 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307272062 2023-02-13 16:54:36,781 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307272471 2023-02-13 16:54:36,781 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307272264 2023-02-13 16:54:36,781 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307272693 2023-02-13 16:54:36,782 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307272471 2023-02-13 16:54:36,782 INFO [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,782 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307272693 2023-02-13 16:54:36,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-13 16:54:36,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 11ec0bb6bbbf4d006fe9a7fb93bd9046, server=jenkins-hbase4.apache.org,34717,1676307263076 in 255 msec 2023-02-13 16:54:36,785 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1 2023-02-13 16:54:36,785 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1 2023-02-13 16:54:36,786 INFO [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4cc157038f0cfa34e35dda26617b6793 columnFamilyName cf1 2023-02-13 16:54:36,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=11ec0bb6bbbf4d006fe9a7fb93bd9046, ASSIGN in 421 msec 2023-02-13 16:54:36,799 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] throttle.PressureAwareThroughputController(145): 11ec0bb6bbbf4d006fe9a7fb93bd9046#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:36,802 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de-top 2023-02-13 16:54:36,809 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5-top 2023-02-13 16:54:36,811 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] throttle.PressureAwareThroughputController(145): 11ec0bb6bbbf4d006fe9a7fb93bd9046#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:36,823 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b-top 2023-02-13 16:54:36,828 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4-top 2023-02-13 16:54:36,835 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/.tmp/cf2/71da2a67fb294397bdd1ed87dca6ae36 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/71da2a67fb294397bdd1ed87dca6ae36 2023-02-13 16:54:36,836 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c-top 2023-02-13 16:54:36,841 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281-top 2023-02-13 16:54:36,844 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/.tmp/cf1/5f2e4d8e99934ad49720628881b1505c as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/5f2e4d8e99934ad49720628881b1505c 2023-02-13 16:54:36,844 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2 of 11ec0bb6bbbf4d006fe9a7fb93bd9046 into 71da2a67fb294397bdd1ed87dca6ae36(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:36,844 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 11ec0bb6bbbf4d006fe9a7fb93bd9046: 2023-02-13 16:54:36,844 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046., storeName=11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2, priority=9, startTime=1676307276775; duration=0sec 2023-02-13 16:54:36,844 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:36,847 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307-top 2023-02-13 16:54:36,847 INFO [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(310): Store=4cc157038f0cfa34e35dda26617b6793/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:36,847 INFO [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,848 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2 2023-02-13 16:54:36,848 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2 2023-02-13 16:54:36,848 INFO [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4cc157038f0cfa34e35dda26617b6793 columnFamilyName cf2 2023-02-13 16:54:36,850 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1 of 11ec0bb6bbbf4d006fe9a7fb93bd9046 into 5f2e4d8e99934ad49720628881b1505c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:36,850 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 11ec0bb6bbbf4d006fe9a7fb93bd9046: 2023-02-13 16:54:36,850 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046., storeName=11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1, priority=9, startTime=1676307276774; duration=0sec 2023-02-13 16:54:36,850 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:36,862 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43-top 2023-02-13 16:54:36,865 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4-top 2023-02-13 16:54:36,869 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8-top 2023-02-13 16:54:36,873 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e-top 2023-02-13 16:54:36,877 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2-top 2023-02-13 16:54:36,882 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870-top 2023-02-13 16:54:36,886 DEBUG [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf-top 2023-02-13 16:54:36,886 INFO [StoreOpener-4cc157038f0cfa34e35dda26617b6793-1] regionserver.HStore(310): Store=4cc157038f0cfa34e35dda26617b6793/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:36,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:36,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:36,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4cc157038f0cfa34e35dda26617b6793; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73781401, jitterRate=0.09942854940891266}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:36,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4cc157038f0cfa34e35dda26617b6793: Running coprocessor pre-open hook at 1676307276780Writing region info on filesystem at 1676307276780Initializing all the Stores at 1676307276781 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307276781Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307276782 (+1 ms)Cleaning up temporary data from old regions at 1676307276888 (+106 ms)Cleaning up detritus from prior splits at 1676307276889 (+1 ms)Running coprocessor post-open hooks at 1676307276891 (+2 ms)Region opened successfully at 1676307276891 2023-02-13 16:54:36,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793., pid=33, masterSystemTime=1676307276673 2023-02-13 16:54:36,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:36,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:54:36,893 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:36,893 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:36,893 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:36,894 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1912): 4cc157038f0cfa34e35dda26617b6793/cf1 is initiating minor compaction (all files) 2023-02-13 16:54:36,894 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:36,894 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4cc157038f0cfa34e35dda26617b6793/cf1 in testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:36,894 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1912): 4cc157038f0cfa34e35dda26617b6793/cf2 is initiating minor compaction (all files) 2023-02-13 16:54:36,894 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4cc157038f0cfa34e35dda26617b6793/cf2 in testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:36,894 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b-top] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/.tmp, totalSize=34.9 K 2023-02-13 16:54:36,894 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43-top] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/.tmp, totalSize=34.9 K 2023-02-13 16:54:36,895 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307271464 2023-02-13 16:54:36,895 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307271464 2023-02-13 16:54:36,895 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307271659 2023-02-13 16:54:36,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:36,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:36,896 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307271659 2023-02-13 16:54:36,896 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307271859 2023-02-13 16:54:36,896 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=4cc157038f0cfa34e35dda26617b6793, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:36,896 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307271859 2023-02-13 16:54:36,896 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307276896"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307276896"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307276896"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307276896"}]},"ts":"1676307276896"} 2023-02-13 16:54:36,897 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676307272062 2023-02-13 16:54:36,897 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676307272062 2023-02-13 16:54:36,897 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676307272264 2023-02-13 16:54:36,897 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676307272264 2023-02-13 16:54:36,898 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676307272471 2023-02-13 16:54:36,898 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676307272471 2023-02-13 16:54:36,898 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307272693 2023-02-13 16:54:36,899 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307272693 2023-02-13 16:54:36,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-13 16:54:36,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 4cc157038f0cfa34e35dda26617b6793, server=jenkins-hbase4.apache.org,34717,1676307263076 in 377 msec 2023-02-13 16:54:36,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-13 16:54:36,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4cc157038f0cfa34e35dda26617b6793, ASSIGN in 539 msec 2023-02-13 16:54:36,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 in 3.9480 sec 2023-02-13 16:54:36,916 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] throttle.PressureAwareThroughputController(145): 4cc157038f0cfa34e35dda26617b6793#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:36,920 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4cc157038f0cfa34e35dda26617b6793#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:36,934 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/.tmp/cf2/18b9422f21aa49ff823aeb1ba0b2a931 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/18b9422f21aa49ff823aeb1ba0b2a931 2023-02-13 16:54:36,940 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4cc157038f0cfa34e35dda26617b6793/cf2 of 4cc157038f0cfa34e35dda26617b6793 into 18b9422f21aa49ff823aeb1ba0b2a931(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:36,940 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4cc157038f0cfa34e35dda26617b6793: 2023-02-13 16:54:36,940 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793., storeName=4cc157038f0cfa34e35dda26617b6793/cf2, priority=9, startTime=1676307276893; duration=0sec 2023-02-13 16:54:36,941 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:36,947 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/.tmp/cf1/ffb73f47306649fa834786ba4f8394c2 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/ffb73f47306649fa834786ba4f8394c2 2023-02-13 16:54:36,953 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4cc157038f0cfa34e35dda26617b6793/cf1 of 4cc157038f0cfa34e35dda26617b6793 into ffb73f47306649fa834786ba4f8394c2(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:36,953 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4cc157038f0cfa34e35dda26617b6793: 2023-02-13 16:54:36,953 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793., storeName=4cc157038f0cfa34e35dda26617b6793/cf1, priority=9, startTime=1676307276892; duration=0sec 2023-02-13 16:54:36,953 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:37,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34717] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:48874 deadline: 1676307337108, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea. is not online on jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:37,220 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 16:54:37,228 INFO [Listener at localhost/38547] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 16:54:37,229 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 16:54:37,245 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:37,247 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-13 16:54:37,247 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-13 16:54:37,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:37,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 16:54:37,252 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307277251"}]},"ts":"1676307277251"} 2023-02-13 16:54:37,253 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-13 16:54:37,255 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-13 16:54:37,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=11ec0bb6bbbf4d006fe9a7fb93bd9046, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4cc157038f0cfa34e35dda26617b6793, UNASSIGN}] 2023-02-13 16:54:37,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4cc157038f0cfa34e35dda26617b6793, UNASSIGN 2023-02-13 16:54:37,257 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=4cc157038f0cfa34e35dda26617b6793, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:37,257 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307277257"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307277257"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307277257"}]},"ts":"1676307277257"} 2023-02-13 16:54:37,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 4cc157038f0cfa34e35dda26617b6793, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:37,261 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=11ec0bb6bbbf4d006fe9a7fb93bd9046, UNASSIGN 2023-02-13 16:54:37,262 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=11ec0bb6bbbf4d006fe9a7fb93bd9046, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:37,262 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307277262"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307277262"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307277262"}]},"ts":"1676307277262"} 2023-02-13 16:54:37,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 11ec0bb6bbbf4d006fe9a7fb93bd9046, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:37,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 16:54:37,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:37,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 11ec0bb6bbbf4d006fe9a7fb93bd9046, disabling compactions & flushes 2023-02-13 16:54:37,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:37,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:37,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. after waiting 0 ms 2023-02-13 16:54:37,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:37,422 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b-bottom] to archive 2023-02-13 16:54:37,424 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:37,425 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,427 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,428 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,429 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,430 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,432 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,433 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,434 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43-bottom] to archive 2023-02-13 16:54:37,435 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:37,437 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,438 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,439 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,440 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,441 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,442 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,444 DEBUG [StoreCloser-testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:54:37,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:37,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046. 2023-02-13 16:54:37,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 11ec0bb6bbbf4d006fe9a7fb93bd9046: Waiting for close lock at 1676307277411Running coprocessor pre-close hooks at 1676307277411Disabling compacts and flushes for region at 1676307277411Disabling writes for close at 1676307277411Writing region close event to WAL at 1676307277445 (+34 ms)Running coprocessor post-close hooks at 1676307277448 (+3 ms)Closed at 1676307277449 (+1 ms) 2023-02-13 16:54:37,450 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:37,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:37,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4cc157038f0cfa34e35dda26617b6793, disabling compactions & flushes 2023-02-13 16:54:37,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:37,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:37,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. after waiting 0 ms 2023-02-13 16:54:37,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:37,451 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=11ec0bb6bbbf4d006fe9a7fb93bd9046, regionState=CLOSED 2023-02-13 16:54:37,452 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307277451"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307277451"}]},"ts":"1676307277451"} 2023-02-13 16:54:37,452 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b-top] to archive 2023-02-13 16:54:37,453 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:37,455 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-13 16:54:37,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 11ec0bb6bbbf4d006fe9a7fb93bd9046, server=jenkins-hbase4.apache.org,34717,1676307263076 in 190 msec 2023-02-13 16:54:37,457 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=11ec0bb6bbbf4d006fe9a7fb93bd9046, UNASSIGN in 200 msec 2023-02-13 16:54:37,458 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,459 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,461 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,462 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,464 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,466 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43-top] to archive 2023-02-13 16:54:37,466 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:37,468 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,469 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,470 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,472 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,473 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,474 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,475 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:54:37,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:37,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793. 2023-02-13 16:54:37,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4cc157038f0cfa34e35dda26617b6793: Waiting for close lock at 1676307277451Running coprocessor pre-close hooks at 1676307277451Disabling compacts and flushes for region at 1676307277451Disabling writes for close at 1676307277451Writing region close event to WAL at 1676307277477 (+26 ms)Running coprocessor post-close hooks at 1676307277480 (+3 ms)Closed at 1676307277480 2023-02-13 16:54:37,482 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:37,482 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=4cc157038f0cfa34e35dda26617b6793, regionState=CLOSED 2023-02-13 16:54:37,482 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307277482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307277482"}]},"ts":"1676307277482"} 2023-02-13 16:54:37,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-13 16:54:37,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 4cc157038f0cfa34e35dda26617b6793, server=jenkins-hbase4.apache.org,34717,1676307263076 in 225 msec 2023-02-13 16:54:37,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-13 16:54:37,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4cc157038f0cfa34e35dda26617b6793, UNASSIGN in 229 msec 2023-02-13 16:54:37,488 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307277488"}]},"ts":"1676307277488"} 2023-02-13 16:54:37,489 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-13 16:54:37,491 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-13 16:54:37,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 245 msec 2023-02-13 16:54:37,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 16:54:37,554 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-13 16:54:37,555 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-13 16:54:37,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:37,558 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:37,558 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:37,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 16:54:37,565 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,565 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:37,565 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:37,568 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/recovered.edits] 2023-02-13 16:54:37,568 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/recovered.edits] 2023-02-13 16:54:37,568 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/recovered.edits] 2023-02-13 16:54:37,573 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/0d036ec9f8b14c7aa964d7fa52faf3de 2023-02-13 16:54:37,573 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/.regioninfo to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/.regioninfo 2023-02-13 16:54:37,573 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/.regioninfo to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/.regioninfo 2023-02-13 16:54:37,574 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/1eeb75e7149d43e2badffd59df6be7d5 2023-02-13 16:54:37,576 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/85b851389d8d4e5e9a20a801a85d064b 2023-02-13 16:54:37,577 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,577 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/0d036ec9f8b14c7aa964d7fa52faf3de.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,578 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/8efe1e3861db401591b145900ec85de4 2023-02-13 16:54:37,579 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,579 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/1eeb75e7149d43e2badffd59df6be7d5.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,579 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/a02e61f2ddb14cacafccac50add8ac4c 2023-02-13 16:54:37,581 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,581 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/85b851389d8d4e5e9a20a801a85d064b.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,581 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/aa0a8f0505554105be3aa6029bdd2281 2023-02-13 16:54:37,582 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,583 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/8efe1e3861db401591b145900ec85de4.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,583 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf1/b9b4c73c64b84b5aadda40c165ac3307 2023-02-13 16:54:37,584 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,585 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/a02e61f2ddb14cacafccac50add8ac4c.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,586 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,587 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/aa0a8f0505554105be3aa6029bdd2281.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,588 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1dde9df079304e60a4f8753ae72a4c43 2023-02-13 16:54:37,589 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,589 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf1/b9b4c73c64b84b5aadda40c165ac3307.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,591 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/1fac80fd9837409e9bf92f6acedb3be4 2023-02-13 16:54:37,593 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/cddceb83e7964154a2688e1751c7c3c8 2023-02-13 16:54:37,594 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,594 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/1dde9df079304e60a4f8753ae72a4c43.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,595 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/d2842e91777c42b1a9826f67c663896e 2023-02-13 16:54:37,596 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,597 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/1fac80fd9837409e9bf92f6acedb3be4.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,598 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ea148b6afa9d4fae941a628f2a6f8ce2 2023-02-13 16:54:37,599 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,599 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/cddceb83e7964154a2688e1751c7c3c8.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,601 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/ec1932c9a6a549fc869dd2cda2526870 2023-02-13 16:54:37,601 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,602 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/d2842e91777c42b1a9826f67c663896e.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,603 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/cf2/fd29c58bdc044b5693e1d460d1b4d8bf 2023-02-13 16:54:37,604 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,604 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ea148b6afa9d4fae941a628f2a6f8ce2.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,606 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,607 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/ec1932c9a6a549fc869dd2cda2526870.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,607 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/recovered.edits/85.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea/recovered.edits/85.seqid 2023-02-13 16:54:37,608 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,608 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,609 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/4cc157038f0cfa34e35dda26617b6793/cf2/fd29c58bdc044b5693e1d460d1b4d8bf.7c340e1b244f9866f9e7640b1e7f78ea 2023-02-13 16:54:37,612 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/ffb73f47306649fa834786ba4f8394c2 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf1/ffb73f47306649fa834786ba4f8394c2 2023-02-13 16:54:37,612 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/5f2e4d8e99934ad49720628881b1505c to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf1/5f2e4d8e99934ad49720628881b1505c 2023-02-13 16:54:37,615 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/18b9422f21aa49ff823aeb1ba0b2a931 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/cf2/18b9422f21aa49ff823aeb1ba0b2a931 2023-02-13 16:54:37,615 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/71da2a67fb294397bdd1ed87dca6ae36 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/cf2/71da2a67fb294397bdd1ed87dca6ae36 2023-02-13 16:54:37,618 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793/recovered.edits/90.seqid 2023-02-13 16:54:37,618 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046/recovered.edits/90.seqid 2023-02-13 16:54:37,619 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:37,619 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitWithoutPONR/11ec0bb6bbbf4d006fe9a7fb93bd9046 2023-02-13 16:54:37,619 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 16:54:37,622 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:37,625 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-13 16:54:37,627 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-13 16:54:37,629 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:37,629 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-13 16:54:37,629 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307277629"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:37,629 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307277629"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:37,629 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307277629"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:37,631 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 16:54:37,631 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7c340e1b244f9866f9e7640b1e7f78ea, NAME => 'testSplitWithoutPONR,,1676307270959.7c340e1b244f9866f9e7640b1e7f78ea.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 11ec0bb6bbbf4d006fe9a7fb93bd9046, NAME => 'testSplitWithoutPONR,,1676307272957.11ec0bb6bbbf4d006fe9a7fb93bd9046.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 4cc157038f0cfa34e35dda26617b6793, NAME => 'testSplitWithoutPONR,41,1676307272957.4cc157038f0cfa34e35dda26617b6793.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 16:54:37,631 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-13 16:54:37,631 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307277631"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:37,632 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-13 16:54:37,635 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:37,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 80 msec 2023-02-13 16:54:37,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 16:54:37,660 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-13 16:54:37,675 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=400 (was 392) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:46920 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:34717-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53834 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53858 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:51648 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:55576 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26bb8ba6-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53872 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:46932 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:51636 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:52736 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53846 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53844 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:52414 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53838 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:46974 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:46960 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26bb8ba6-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:55774 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=857 (was 841) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 260), ProcessCount=168 (was 168), AvailableMemoryMB=1354 (was 1537) 2023-02-13 16:54:37,690 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400, OpenFileDescriptor=857, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=168, AvailableMemoryMB=1353 2023-02-13 16:54:37,690 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:37,693 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:54:37,694 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:54:37,695 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:37,697 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:37,698 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:37,699 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:37,700 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547 empty. 2023-02-13 16:54:37,700 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:37,700 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 16:54:37,710 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:37,712 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1ad9073aa1705b085f9e78eda1933547, NAME => 'testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:54:37,720 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:37,720 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 1ad9073aa1705b085f9e78eda1933547, disabling compactions & flushes 2023-02-13 16:54:37,720 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:37,720 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:37,720 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. after waiting 0 ms 2023-02-13 16:54:37,720 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:37,720 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:37,720 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 1ad9073aa1705b085f9e78eda1933547: Waiting for close lock at 1676307277720Disabling compacts and flushes for region at 1676307277720Disabling writes for close at 1676307277720Writing region close event to WAL at 1676307277720Closed at 1676307277720 2023-02-13 16:54:37,723 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:37,723 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307277723"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307277723"}]},"ts":"1676307277723"} 2023-02-13 16:54:37,725 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:37,726 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:37,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307277726"}]},"ts":"1676307277726"} 2023-02-13 16:54:37,727 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 16:54:37,729 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:54:37,730 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:54:37,730 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:54:37,730 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:54:37,730 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:54:37,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN}] 2023-02-13 16:54:37,732 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:37,733 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:37,883 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:54:37,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:37,884 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307277884"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307277884"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307277884"}]},"ts":"1676307277884"} 2023-02-13 16:54:37,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:38,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1ad9073aa1705b085f9e78eda1933547, NAME => 'testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:38,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:38,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:38,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:38,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:38,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:38,043 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:38,044 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1 2023-02-13 16:54:38,044 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1 2023-02-13 16:54:38,045 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ad9073aa1705b085f9e78eda1933547 columnFamilyName cf1 2023-02-13 16:54:38,045 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(310): Store=1ad9073aa1705b085f9e78eda1933547/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:38,045 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:38,046 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2 2023-02-13 16:54:38,046 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2 2023-02-13 16:54:38,047 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ad9073aa1705b085f9e78eda1933547 columnFamilyName cf2 2023-02-13 16:54:38,047 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(310): Store=1ad9073aa1705b085f9e78eda1933547/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:38,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:38,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:38,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:38,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:38,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:38,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1ad9073aa1705b085f9e78eda1933547; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68824375, jitterRate=0.025563105940818787}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:38,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1ad9073aa1705b085f9e78eda1933547: Running coprocessor pre-open hook at 1676307278042Writing region info on filesystem at 1676307278042Initializing all the Stores at 1676307278042Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307278043 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307278043Cleaning up temporary data from old regions at 1676307278049 (+6 ms)Cleaning up detritus from prior splits at 1676307278049Running coprocessor post-open hooks at 1676307278053 (+4 ms)Region opened successfully at 1676307278053 2023-02-13 16:54:38,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547., pid=43, masterSystemTime=1676307278038 2023-02-13 16:54:38,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,056 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,056 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307278056"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307278056"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307278056"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307278056"}]},"ts":"1676307278056"} 2023-02-13 16:54:38,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-13 16:54:38,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 in 172 msec 2023-02-13 16:54:38,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-13 16:54:38,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN in 328 msec 2023-02-13 16:54:38,062 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:38,062 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307278062"}]},"ts":"1676307278062"} 2023-02-13 16:54:38,063 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 16:54:38,069 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:38,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 375 msec 2023-02-13 16:54:38,204 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:38,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 16:54:38,206 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,206 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,206 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 16:54:38,206 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:54:38,207 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,207 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,208 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,208 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,208 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,208 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,208 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,208 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:38,210 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,210 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,210 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. started... 2023-02-13 16:54:38,211 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ad9073aa1705b085f9e78eda1933547 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 16:54:38,212 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,213 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,213 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,213 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,213 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,214 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,226 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/36071240f15943f49df2b7a68151e3dd 2023-02-13 16:54:38,242 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/b357314c7faf4fc1ab1e900060fae146 2023-02-13 16:54:38,247 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/36071240f15943f49df2b7a68151e3dd as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd 2023-02-13 16:54:38,251 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:54:38,252 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/b357314c7faf4fc1ab1e900060fae146 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146 2023-02-13 16:54:38,255 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:54:38,256 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 1ad9073aa1705b085f9e78eda1933547 in 45ms, sequenceid=6, compaction requested=false 2023-02-13 16:54:38,256 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,257 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ad9073aa1705b085f9e78eda1933547: 2023-02-13 16:54:38,257 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,257 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:38,257 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:38,257 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,257 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,257 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,260 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 16:54:38,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79ef8ef9[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:38,262 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:54:38,262 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,264 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:38,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:38,266 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:38,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:54:38,270 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 16:54:38,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:38,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:38,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 16:54:38,270 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:38,273 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,273 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,273 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,273 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,273 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,273 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,273 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,273 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,274 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,275 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,277 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,277 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,277 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,280 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,280 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,281 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:38,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,281 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,281 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:38,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,281 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,281 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:38,281 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,282 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,282 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,382 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:38,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:38,399 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:38,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:38,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 16:54:38,401 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,401 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,401 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 16:54:38,401 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:54:38,401 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,401 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,402 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,402 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,402 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,402 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,403 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,403 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:38,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,405 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,405 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,405 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,405 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,405 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,405 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,406 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,406 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,407 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,409 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,409 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,409 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,409 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,409 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,412 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:38,412 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:38,412 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,412 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. started... 2023-02-13 16:54:38,412 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ad9073aa1705b085f9e78eda1933547 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:38,423 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/e4a7f2594e7f4149b2dd18583cae833d 2023-02-13 16:54:38,441 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 2023-02-13 16:54:38,446 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/e4a7f2594e7f4149b2dd18583cae833d as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d 2023-02-13 16:54:38,451 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:54:38,452 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 2023-02-13 16:54:38,456 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:54:38,457 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ad9073aa1705b085f9e78eda1933547 in 45ms, sequenceid=19, compaction requested=false 2023-02-13 16:54:38,457 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ad9073aa1705b085f9e78eda1933547: 2023-02-13 16:54:38,457 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:38,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:38,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,459 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 16:54:38,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@440abef2[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:38,461 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:54:38,461 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,462 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,462 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,462 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,462 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,463 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,463 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,463 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:38,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,463 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:38,463 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,463 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,463 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:38,463 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,463 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,463 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:38,464 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,464 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,464 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,468 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,468 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,468 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,469 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,469 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,469 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,469 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:38,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:54:38,472 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 16:54:38,472 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:38,472 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:38,472 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 16:54:38,472 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:38,473 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,473 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,473 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,473 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,473 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,474 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,473 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,473 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,474 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,475 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,477 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,477 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,477 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,480 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,480 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,480 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,480 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,480 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,480 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,480 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,480 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,480 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,480 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,480 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:38,480 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:38,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:38,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,481 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,581 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:38,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:38,597 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:38,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:38,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 16:54:38,598 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,598 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,598 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 16:54:38,598 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:54:38,599 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,599 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,601 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,601 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,601 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,601 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,602 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,602 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,602 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,603 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,603 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,604 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:38,604 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:38,604 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,604 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. started... 2023-02-13 16:54:38,604 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ad9073aa1705b085f9e78eda1933547 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:38,604 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,605 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,605 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,605 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,606 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,606 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,613 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/1149efdf750b4b69bd3108b446b27289 2023-02-13 16:54:38,631 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/27c693f57584430d99b495942e0f67e6 2023-02-13 16:54:38,635 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/1149efdf750b4b69bd3108b446b27289 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289 2023-02-13 16:54:38,639 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:54:38,640 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/27c693f57584430d99b495942e0f67e6 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6 2023-02-13 16:54:38,644 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:54:38,645 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ad9073aa1705b085f9e78eda1933547 in 41ms, sequenceid=32, compaction requested=true 2023-02-13 16:54:38,645 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ad9073aa1705b085f9e78eda1933547: 2023-02-13 16:54:38,645 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,645 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:38,645 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:38,645 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,645 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,645 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,646 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,646 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,647 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,647 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 16:54:38,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@308267df[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:38,649 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:54:38,649 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,651 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,651 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,651 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,651 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:38,651 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:38,651 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,651 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,651 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:38,651 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,652 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,652 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:38,653 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,653 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,653 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,653 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,653 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,653 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,653 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,654 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,654 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,654 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:38,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:54:38,657 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 16:54:38,657 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:38,657 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:38,657 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 16:54:38,657 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:38,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,660 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,661 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,662 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,664 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,664 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,664 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,670 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,670 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:38,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,670 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,670 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:38,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,671 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,671 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,671 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,671 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,671 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,671 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,671 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,771 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:38,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:38,786 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:38,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:38,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 16:54:38,788 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,788 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,788 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 16:54:38,788 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:54:38,788 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,788 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,789 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,789 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,789 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,789 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,790 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,790 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:38,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,791 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,791 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,791 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,791 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,791 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,791 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,791 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,792 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:38,792 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:38,792 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,792 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. started... 2023-02-13 16:54:38,792 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ad9073aa1705b085f9e78eda1933547 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:38,794 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,794 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,794 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,794 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,794 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,794 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,802 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/0047f9f32cfc4b51b97de82713fe1c68 2023-02-13 16:54:38,815 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/cb417272edfb4d9cb5cea06ff823a703 2023-02-13 16:54:38,819 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/0047f9f32cfc4b51b97de82713fe1c68 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68 2023-02-13 16:54:38,823 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:54:38,824 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/cb417272edfb4d9cb5cea06ff823a703 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703 2023-02-13 16:54:38,827 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:54:38,828 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ad9073aa1705b085f9e78eda1933547 in 36ms, sequenceid=45, compaction requested=true 2023-02-13 16:54:38,828 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ad9073aa1705b085f9e78eda1933547: 2023-02-13 16:54:38,829 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,829 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:38,829 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:38,829 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,829 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,829 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,830 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,830 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,831 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,831 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 16:54:38,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c8ce796[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:38,832 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:54:38,832 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,833 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,833 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,833 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,834 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,834 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,834 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:38,834 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,834 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:38,834 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,834 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,834 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:38,834 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:38,834 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:38,834 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:38,835 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,835 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,835 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,836 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,836 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:38,837 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,836 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,837 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,837 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,837 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:38,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:38,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:54:38,840 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 16:54:38,840 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:38,840 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:38,840 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 16:54:38,840 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:38,841 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,841 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,841 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,841 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,841 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,841 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,842 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,842 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,842 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,843 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,844 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,845 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,845 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:38,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,847 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,847 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,847 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,847 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,847 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,847 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,847 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,847 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,848 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:38,847 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,847 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,848 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:38,847 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,848 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:38,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,848 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,848 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,848 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:38,848 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,848 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,849 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:38,849 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,849 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,948 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:38,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:38,964 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:38,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:38,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 16:54:38,966 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 16:54:38,966 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,966 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 16:54:38,966 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:54:38,967 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,967 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,968 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,968 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,968 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,968 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,968 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:38,968 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:38,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,969 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,969 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,969 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:38,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:38,970 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:38,970 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. started... 2023-02-13 16:54:38,971 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ad9073aa1705b085f9e78eda1933547 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:38,972 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,982 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/f30e19700bca4b90a11eebe72969f448 2023-02-13 16:54:38,997 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f 2023-02-13 16:54:39,001 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/f30e19700bca4b90a11eebe72969f448 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448 2023-02-13 16:54:39,005 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:54:39,005 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f 2023-02-13 16:54:39,009 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:54:39,010 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ad9073aa1705b085f9e78eda1933547 in 39ms, sequenceid=58, compaction requested=true 2023-02-13 16:54:39,010 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ad9073aa1705b085f9e78eda1933547: 2023-02-13 16:54:39,010 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:39,011 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:39,012 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,012 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:39,012 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,012 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:39,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 16:54:39,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39912eb8[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:39,014 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:54:39,014 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,015 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,015 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,015 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,015 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,015 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,015 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:39,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:39,015 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:39,018 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:39,018 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,018 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:39,021 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,021 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:39,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:39,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:39,021 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,021 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:39,022 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:39,022 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,022 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:39,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:54:39,025 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 16:54:39,025 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:39,025 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:39,025 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 16:54:39,025 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:39,027 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,027 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,027 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,028 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,028 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,028 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,028 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,028 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:39,028 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,029 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,030 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,030 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,031 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,033 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,033 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,033 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,033 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,033 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,033 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,033 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,033 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,033 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 16:54:39,034 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:39,033 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,033 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,034 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:39,034 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,034 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:39,034 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,034 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,034 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,035 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,035 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,035 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,035 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,134 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:39,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:39,149 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:39,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:39,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 16:54:39,151 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 16:54:39,151 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:39,151 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 16:54:39,151 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:54:39,151 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,151 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,152 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,152 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,153 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,153 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,153 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:39,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,153 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,153 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:39,154 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:39,154 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:39,154 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:39,154 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:39,154 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:39,154 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 16:54:39,154 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:39,155 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 16:54:39,155 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:39,155 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 16:54:39,154 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:39,155 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:39,155 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:39,155 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:54:39,155 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:39,155 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:39,156 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:39,156 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:39,156 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:39,156 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. started... 2023-02-13 16:54:39,156 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ad9073aa1705b085f9e78eda1933547 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:39,156 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,156 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,156 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:39,158 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,158 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,158 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:39,165 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/d381b11f63164380bbc8ad286993df9f 2023-02-13 16:54:39,178 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/142664c8dc1342f0ac4fa30ff8081f88 2023-02-13 16:54:39,183 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/d381b11f63164380bbc8ad286993df9f as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f 2023-02-13 16:54:39,188 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:54:39,189 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/142664c8dc1342f0ac4fa30ff8081f88 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88 2023-02-13 16:54:39,193 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:54:39,194 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ad9073aa1705b085f9e78eda1933547 in 38ms, sequenceid=71, compaction requested=true 2023-02-13 16:54:39,194 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ad9073aa1705b085f9e78eda1933547: 2023-02-13 16:54:39,194 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:39,194 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:39,194 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:39,194 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,194 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 16:54:39,194 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 16:54:39,196 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,196 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:39,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:39,196 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:39,196 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:39,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:39,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:39,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 16:54:39,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28fdfa56[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:39,198 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:54:39,198 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,199 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,199 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,199 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,199 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,199 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:39,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,199 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:39,199 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:39,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,199 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:39,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 16:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:39,201 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:39,201 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,201 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:39,202 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:39,202 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,202 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 16:54:39,202 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:39,202 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,203 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:39,203 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 16:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:39,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:39,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:54:39,205 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 16:54:39,206 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:39,206 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:39,206 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 16:54:39,206 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:39,207 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:39,207 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,207 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:39,207 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,208 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,207 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:39,208 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,209 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 16:54:39,210 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,210 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,211 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,216 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,216 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,216 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,216 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,216 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,216 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,216 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,217 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,216 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 16:54:39,217 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:39,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,216 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,216 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,217 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 16:54:39,217 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,217 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:39,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,217 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,217 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,217 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:39,217 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:39,218 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:39,218 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 16:54:39,218 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 16:54:39,318 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:39,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:39,578 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 16:54:39,579 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:39,580 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,582 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,582 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,582 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,582 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:39,582 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:39,583 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:39,831 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,831 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:39,831 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:39,831 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:39,831 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-13 16:54:39,831 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:39,831 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:39,831 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:39,832 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:39,832 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:39,833 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 16:54:39,833 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:39,834 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:39,834 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:39,834 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:39,834 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:39,835 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:39,835 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:39,835 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:39,835 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:39,836 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:39,836 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:39,836 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:39,836 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:39,836 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,837 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:39,837 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,837 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,837 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,837 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:39,837 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:39,838 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:39,838 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307309838 2023-02-13 16:54:39,838 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:39,840 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,842 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:39,842 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:39,842 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:39,842 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307339842 2023-02-13 16:54:39,842 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307399842 2023-02-13 16:54:39,842 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:39,843 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:39,846 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:39,846 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:39,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:39,846 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:40,093 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,093 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:40,093 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:40,093 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:40,094 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:40,094 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:40,094 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:40,094 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:40,094 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:40,095 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:40,096 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 16:54:40,096 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:40,097 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:40,097 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:40,097 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:40,097 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,098 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:40,098 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,098 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,098 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,099 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:40,099 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,099 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,099 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:40,099 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,099 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:40,100 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,100 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,100 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,100 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:40,100 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:40,101 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:40,101 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307310101 2023-02-13 16:54:40,101 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:40,103 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,106 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:40,106 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:40,106 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:40,106 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307340106 2023-02-13 16:54:40,106 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307400106 2023-02-13 16:54:40,106 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:40,107 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:40,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN}] 2023-02-13 16:54:40,109 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:40,109 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:40,109 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:40,357 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,357 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:40,357 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:40,357 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:40,358 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:40,358 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:40,358 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:40,358 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:40,359 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:40,359 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:40,360 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 16:54:40,361 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:40,361 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:40,362 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:40,362 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:40,362 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,363 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:40,363 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,363 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,363 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,364 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:40,364 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,365 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,365 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:40,365 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,366 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:40,366 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,366 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,366 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,366 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 16:54:40,367 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:40,367 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:40,368 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307310368 2023-02-13 16:54:40,369 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:40,371 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:40,372 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:40,372 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:40,372 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307340372 2023-02-13 16:54:40,372 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307400372 2023-02-13 16:54:40,372 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:54:40,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN}] 2023-02-13 16:54:40,373 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:40,374 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:40,375 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:40,375 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307280375"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307280375"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307280375"}]},"ts":"1676307280375"} 2023-02-13 16:54:40,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:40,377 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:40,377 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:40,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:40,377 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:40,622 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,623 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:40,623 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:40,623 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:40,623 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:40,623 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:40,623 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:40,623 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:40,623 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:40,624 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:40,625 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:54:40,625 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 16:54:40,625 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:40,626 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:40,626 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:40,626 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:40,626 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,627 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:40,627 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,627 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,627 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,627 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:40,628 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,628 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,628 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:40,629 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,629 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:40,629 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:40,629 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,629 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,629 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,629 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:40,629 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:40,629 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 16:54:40,630 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:40,630 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:40,631 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547 to restore RIT 2023-02-13 16:54:40,632 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307310632 2023-02-13 16:54:40,633 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:40,633 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:40,635 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:40,635 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:40,635 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:40,635 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307340635 2023-02-13 16:54:40,635 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307400635 2023-02-13 16:54:40,635 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:54:40,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:40,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307280636"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307280636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307280636"}]},"ts":"1676307280636"} 2023-02-13 16:54:40,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:40,637 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:40,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:40,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1ad9073aa1705b085f9e78eda1933547, disabling compactions & flushes 2023-02-13 16:54:40,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:40,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:40,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. after waiting 0 ms 2023-02-13 16:54:40,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:40,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1ad9073aa1705b085f9e78eda1933547 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 16:54:40,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/99f1322d133349caa900766fb998ad97 2023-02-13 16:54:40,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/5505df4447a04cc2bb74661b98bcdc4b 2023-02-13 16:54:40,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/99f1322d133349caa900766fb998ad97 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97 2023-02-13 16:54:40,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:54:40,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/5505df4447a04cc2bb74661b98bcdc4b as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b 2023-02-13 16:54:40,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:54:40,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1ad9073aa1705b085f9e78eda1933547 in 39ms, sequenceid=82, compaction requested=true 2023-02-13 16:54:40,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 16:54:40,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:40,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:40,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1ad9073aa1705b085f9e78eda1933547: Waiting for close lock at 1676307280790Running coprocessor pre-close hooks at 1676307280790Disabling compacts and flushes for region at 1676307280790Disabling writes for close at 1676307280790Obtaining lock to block concurrent updates at 1676307280790Preparing flush snapshotting stores in 1ad9073aa1705b085f9e78eda1933547 at 1676307280790Finished memstore snapshotting testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676307280791 (+1 ms)Flushing stores of testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. at 1676307280791Flushing 1ad9073aa1705b085f9e78eda1933547/cf1: creating writer at 1676307280792 (+1 ms)Flushing 1ad9073aa1705b085f9e78eda1933547/cf1: appending metadata at 1676307280794 (+2 ms)Flushing 1ad9073aa1705b085f9e78eda1933547/cf1: closing flushed file at 1676307280794Flushing 1ad9073aa1705b085f9e78eda1933547/cf2: creating writer at 1676307280805 (+11 ms)Flushing 1ad9073aa1705b085f9e78eda1933547/cf2: appending metadata at 1676307280806 (+1 ms)Flushing 1ad9073aa1705b085f9e78eda1933547/cf2: closing flushed file at 1676307280806Flushing 1ad9073aa1705b085f9e78eda1933547/cf1: reopening flushed file at 1676307280818 (+12 ms)Flushing 1ad9073aa1705b085f9e78eda1933547/cf2: reopening flushed file at 1676307280824 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1ad9073aa1705b085f9e78eda1933547 in 39ms, sequenceid=82, compaction requested=true at 1676307280829 (+5 ms)Writing region close event to WAL at 1676307280831 (+2 ms)Running coprocessor post-close hooks at 1676307280835 (+4 ms)Closed at 1676307280835 2023-02-13 16:54:40,837 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:40,838 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED 2023-02-13 16:54:40,838 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307280838"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307280838"}]},"ts":"1676307280838"} 2023-02-13 16:54:40,839 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:40,839 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:40,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:40,840 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:40,886 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,886 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:40,886 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:40,886 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:40,887 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:40,887 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:40,887 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:40,887 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:40,887 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:40,888 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:40,889 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:54:40,889 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 16:54:40,889 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 16:54:40,889 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:40,890 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:40,890 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:40,890 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:40,890 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,890 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:40,891 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,891 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:40,891 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,892 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:40,892 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,892 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:40,892 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:40,893 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,893 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:40,893 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:40,893 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:40,893 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,893 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,893 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:40,893 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:40,893 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:40,893 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:40,894 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 16:54:40,894 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:40,895 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:40,895 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547 to restore RIT 2023-02-13 16:54:40,895 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307310895 2023-02-13 16:54:40,896 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:40,896 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:40,899 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:40,899 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:40,899 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:40,899 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307340899 2023-02-13 16:54:40,899 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307400899 2023-02-13 16:54:40,900 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:54:40,900 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:40,900 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307280900"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307280900"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307280900"}]},"ts":"1676307280900"} 2023-02-13 16:54:40,902 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:40,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 16:54:40,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 in 265 msec 2023-02-13 16:54:40,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:40,905 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:40,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:40,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:41,150 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,150 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:41,150 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:41,150 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:41,153 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:41,153 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:41,153 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:41,154 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:41,154 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:41,154 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:41,156 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:54:41,156 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 16:54:41,156 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:41,156 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:41,157 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:41,157 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:41,157 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:41,158 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:41,158 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:41,158 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:41,158 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:41,159 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:41,159 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:41,159 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:41,160 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:41,160 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:41,161 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,161 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:41,161 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:41,161 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:41,162 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,162 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,162 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,162 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:41,162 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:41,162 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 16:54:41,163 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:41,164 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:41,164 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547 to restore RIT 2023-02-13 16:54:41,168 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307311168 2023-02-13 16:54:41,168 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:41,169 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:41,171 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:41,171 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:41,171 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:41,171 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307341171 2023-02-13 16:54:41,171 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307401171 2023-02-13 16:54:41,172 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:54:41,172 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:41,174 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:41,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 16:54:41,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN in 801 msec 2023-02-13 16:54:41,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:41,181 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:41,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:41,422 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,422 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:41,422 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:41,422 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:41,423 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:41,423 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:41,424 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:41,424 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:41,424 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:41,425 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:41,426 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:54:41,426 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:41,426 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:41,426 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:41,427 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:41,427 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:41,427 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:41,427 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:41,428 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:41,428 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:41,428 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:41,428 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:41,428 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:41,429 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:41,429 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:41,429 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:41,429 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,429 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:41,429 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:41,429 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:41,430 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:41,430 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:41,430 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:41,430 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:41,431 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:41,431 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307311431 2023-02-13 16:54:41,431 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:41,433 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,435 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:41,435 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:41,436 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:41,436 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307341436 2023-02-13 16:54:41,436 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307401436 2023-02-13 16:54:41,436 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:41,436 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,436 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,436 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:41,441 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:41,451 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=1ad9073aa1705b085f9e78eda1933547, threads=14 2023-02-13 16:54:41,453 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,453 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,453 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,453 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,453 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,453 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,453 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,454 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,454 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,456 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,456 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,458 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,459 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,459 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,620 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,644 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,645 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,648 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,676 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,676 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,677 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,679 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,680 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,682 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,683 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,685 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:41,685 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,014 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,014 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 1ad9073aa1705b085f9e78eda1933547 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 16:54:42,057 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:42,058 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:42,058 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:42,180 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:42,186 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,186 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:42,186 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:42,186 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:42,187 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:42,187 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:42,187 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:42,187 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:42,187 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:42,188 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:42,189 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 16:54:42,189 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:42,189 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:42,190 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:42,190 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:42,190 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:42,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:42,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:42,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:42,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:42,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:42,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:42,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:42,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:42,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:42,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:42,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:42,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:42,195 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:42,195 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,195 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,195 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,195 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:42,195 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:42,195 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 16:54:42,196 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:42,197 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:42,199 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307312199 2023-02-13 16:54:42,199 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:42,201 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,203 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:42,203 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:42,204 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:42,204 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307342204 2023-02-13 16:54:42,204 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307402204 2023-02-13 16:54:42,204 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 16:54:42,216 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=1ad9073aa1705b085f9e78eda1933547, threads=14 2023-02-13 16:54:42,217 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,217 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,219 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,219 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,223 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,223 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,224 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,225 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,226 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,227 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,229 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,229 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,230 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,230 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,360 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,360 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,364 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,372 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,383 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,388 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,392 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,392 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,392 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,396 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,399 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,400 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97 for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,712 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,712 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d for region: 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:42,712 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 1ad9073aa1705b085f9e78eda1933547 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 16:54:42,747 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:42,752 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/2b1539d8e8a20fc139b22fb9633eb840/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:54:42,754 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/c7e558972556a4755a23d3e955b540d9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:54:42,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:42,754 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:42,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:42,954 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,954 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:42,955 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:42,955 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:42,956 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:42,956 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:42,956 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:42,956 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:42,956 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:42,957 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:42,957 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 16:54:42,957 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:42,957 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:42,958 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:42,958 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:42,958 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:42,958 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:42,959 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:42,959 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:42,959 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:42,959 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:42,960 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:42,960 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:42,960 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:42,960 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:42,960 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:42,961 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,961 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:42,961 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:42,963 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:42,963 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,963 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,963 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,964 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:42,964 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:42,964 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 16:54:42,964 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:42,965 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:42,965 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307312965 2023-02-13 16:54:42,966 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:42,966 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,968 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:42,968 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:42,969 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:42,969 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307342969 2023-02-13 16:54:42,969 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307402969 2023-02-13 16:54:42,969 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:54:42,972 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:42,974 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:42,974 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:42,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:42,974 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:43,219 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,220 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:43,220 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:43,220 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:43,221 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:43,221 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:43,221 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:43,221 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:43,221 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:43,222 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:43,223 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 16:54:43,223 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:43,223 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:43,223 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:43,223 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:43,223 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:43,224 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:43,224 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:43,225 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:43,225 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:43,225 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:43,225 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:43,225 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:43,225 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:43,226 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:43,226 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:43,226 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,226 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:43,226 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:43,226 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:43,226 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,227 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,227 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,227 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:43,227 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:43,227 DEBUG [Listener at localhost/38547] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,227 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 16:54:43,228 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:43,228 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:43,229 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307313229 2023-02-13 16:54:43,229 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:43,231 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,233 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:43,234 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:43,234 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:43,234 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307343234 2023-02-13 16:54:43,234 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307403234 2023-02-13 16:54:43,234 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:43,234 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:43,247 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:43,247 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 16:54:43,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:43,247 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:43,485 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,485 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:43,485 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:43,485 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:43,486 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:43,486 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:43,486 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:43,486 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:43,486 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:43,487 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:43,488 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 16:54:43,488 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:43,488 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:43,488 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:43,489 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:43,489 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:43,489 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:43,489 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:43,490 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:43,490 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:43,490 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:43,490 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:43,490 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:43,490 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:43,491 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:43,491 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:43,491 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,491 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:43,492 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:43,492 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:43,492 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,492 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,492 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,492 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:43,492 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:43,492 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:43,493 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:43,493 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:43,493 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307313493 2023-02-13 16:54:43,496 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:43,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,498 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:43,498 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:43,498 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:43,498 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307343498 2023-02-13 16:54:43,498 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307403498 2023-02-13 16:54:43,498 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:54:43,499 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:43,501 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:43,501 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 16:54:43,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:43,501 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:43,749 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,750 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:43,750 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:43,750 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:43,750 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:43,750 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:43,750 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:43,750 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:43,750 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:43,751 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:43,752 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 16:54:43,752 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:43,752 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:43,752 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:43,752 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:43,753 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:43,753 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:43,753 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:43,753 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:43,753 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:43,753 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:43,754 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:43,754 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:43,754 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:43,754 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:43,754 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:43,755 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,755 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:43,755 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:43,755 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:43,755 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,755 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,755 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,756 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:43,756 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:43,756 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:43,756 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:43,757 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:43,757 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307313757 2023-02-13 16:54:43,757 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:43,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:43,761 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:43,761 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:43,761 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:43,761 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307343761 2023-02-13 16:54:43,761 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307403761 2023-02-13 16:54:43,761 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:43,761 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:43,764 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:43,764 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 16:54:43,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:43,764 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:44,011 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,012 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:44,012 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:44,012 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:44,012 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:44,012 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:44,012 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:44,012 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:44,012 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:44,013 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:44,014 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 16:54:44,014 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:44,014 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:44,014 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:44,015 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:44,015 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:44,015 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:44,015 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,015 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:44,015 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:44,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,016 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:44,017 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,017 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:44,017 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:44,017 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:44,017 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,017 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,017 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,017 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:44,017 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:44,017 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:44,018 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:44,018 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:44,019 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307314019 2023-02-13 16:54:44,020 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:44,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,022 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:44,023 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:44,023 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:44,023 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307344023 2023-02-13 16:54:44,023 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307404023 2023-02-13 16:54:44,023 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:54:44,023 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:44,026 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN] 2023-02-13 16:54:44,026 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:44,026 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 16:54:44,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:44,026 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:44,273 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,273 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:44,273 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:44,273 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:44,274 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:44,274 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:44,274 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:44,274 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:44,275 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:44,275 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:44,276 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 16:54:44,276 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:54:44,276 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:54:44,276 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 16:54:44,276 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:44,277 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:44,277 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:44,277 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:44,277 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,277 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:44,278 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,278 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,278 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,278 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:44,278 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,278 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,278 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:44,279 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,279 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:44,280 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:44,280 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:44,280 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:44,280 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,280 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,280 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,280 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:44,280 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:44,280 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:44,280 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:44,281 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:44,281 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:44,281 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547 to restore RIT 2023-02-13 16:54:44,284 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307314284 2023-02-13 16:54:44,285 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:44,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,287 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:44,289 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:44,289 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:44,289 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:44,289 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307344289 2023-02-13 16:54:44,289 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307404289 2023-02-13 16:54:44,289 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:44,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:44,294 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:44,295 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 16:54:44,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:44,295 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:44,539 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,540 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:44,540 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:44,540 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:44,541 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:44,541 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:44,541 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:44,541 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:44,541 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:44,542 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:44,542 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:54:44,542 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 16:54:44,543 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:44,543 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:44,543 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:44,543 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:44,543 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,544 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:44,544 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,544 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,544 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,544 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:44,544 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,545 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,545 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:44,545 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,545 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:44,545 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:44,545 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,545 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,546 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,546 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:44,546 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-13 16:54:44,546 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:44,547 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:44,547 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547 to restore RIT 2023-02-13 16:54:44,547 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307314547 2023-02-13 16:54:44,548 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:44,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,550 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:44,551 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:44,551 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:44,552 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:44,552 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307344552 2023-02-13 16:54:44,552 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307404552 2023-02-13 16:54:44,552 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:54:44,552 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:44,554 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:44,554 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 16:54:44,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:44,554 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:44,802 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,802 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:44,802 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:44,802 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:44,803 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:44,803 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:44,803 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:44,803 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:44,803 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:44,804 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:44,805 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 16:54:44,805 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 16:54:44,805 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:44,806 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:44,806 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:44,806 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:44,806 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,806 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:44,807 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,807 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:44,807 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,807 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:44,807 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,807 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:44,808 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:44,808 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,808 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:44,808 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:44,808 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,808 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,808 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,808 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:44,809 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:44,809 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:44,810 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:44,810 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547 to restore RIT 2023-02-13 16:54:44,810 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307314810 2023-02-13 16:54:44,811 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:44,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:44,813 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:44,814 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:44,815 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:44,815 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:44,815 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307344815 2023-02-13 16:54:44,815 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307404815 2023-02-13 16:54:44,815 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:54:44,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:44,817 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:44,817 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 16:54:44,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:44,817 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:45,065 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:45,065 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:45,066 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:45,066 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:45,068 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:45,068 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:45,068 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:45,068 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:45,068 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:45,069 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:45,070 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 16:54:45,070 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 16:54:45,070 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:45,070 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:45,070 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:45,070 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:45,071 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:45,071 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:45,071 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:45,071 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:45,071 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:45,072 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:45,072 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:45,072 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:45,072 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:45,073 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:45,073 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:45,073 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:45,073 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:45,073 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:45,073 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:45,073 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:54:45,074 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 16:54:45,074 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:45,075 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:45,075 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547 to restore RIT 2023-02-13 16:54:45,075 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307315075 2023-02-13 16:54:45,076 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:45,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:45,079 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:45,081 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:45,082 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:45,082 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:45,082 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307345082 2023-02-13 16:54:45,082 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307405082 2023-02-13 16:54:45,082 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 16:54:45,082 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:54:45,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 exec-time=5.5030 sec 2023-02-13 16:54:45,085 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:45,085 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=true 2023-02-13 16:54:45,236 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:54:45,237 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:45,237 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307285237"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307285237"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307285237"}]},"ts":"1676307285237"} 2023-02-13 16:54:45,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:45,239 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:54:45,239 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:45,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:54:45,239 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:54:45,332 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:45,332 WARN [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-13 16:54:45,332 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:54:45,332 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:54:45,333 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:54:45,333 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:54:45,333 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:54:45,333 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:54:45,333 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:54:45,333 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:54:45,334 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:54:45,335 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-13 16:54:45,335 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:54:45,336 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:54:45,336 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:54:45,336 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:54:45,337 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:45,337 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:54:45,337 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:45,337 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:54:45,338 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:45,338 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:54:45,338 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:45,338 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:54:45,339 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:45,339 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:54:45,339 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:45,339 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:54:45,339 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:45,340 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 16:54:45,340 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:54:45,341 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:54:45,341 INFO [Listener at localhost/38547] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547 to restore RIT 2023-02-13 16:54:45,341 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307315341 2023-02-13 16:54:45,342 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:54:45,342 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:54:45,344 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:54:45,344 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=1ad9073aa1705b085f9e78eda1933547, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,34717,1676307263076, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076, openSeqNum=2 2023-02-13 16:54:45,345 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:54:45,345 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307345345 2023-02-13 16:54:45,345 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307405345 2023-02-13 16:54:45,345 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:54:45,345 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:45,345 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307285345"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307285345"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307285345"}]},"ts":"1676307285345"} 2023-02-13 16:54:45,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:45,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:45,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1ad9073aa1705b085f9e78eda1933547, NAME => 'testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:45,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:45,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:45,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:45,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:45,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:45,504 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:45,505 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1 2023-02-13 16:54:45,505 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1 2023-02-13 16:54:45,506 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ad9073aa1705b085f9e78eda1933547 columnFamilyName cf1 2023-02-13 16:54:45,512 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68 2023-02-13 16:54:45,516 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289 2023-02-13 16:54:45,518 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd 2023-02-13 16:54:45,521 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97 2023-02-13 16:54:45,524 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f 2023-02-13 16:54:45,527 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d 2023-02-13 16:54:45,531 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448 2023-02-13 16:54:45,531 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(310): Store=1ad9073aa1705b085f9e78eda1933547/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:45,531 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:45,532 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2 2023-02-13 16:54:45,532 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2 2023-02-13 16:54:45,533 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ad9073aa1705b085f9e78eda1933547 columnFamilyName cf2 2023-02-13 16:54:45,539 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88 2023-02-13 16:54:45,542 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f 2023-02-13 16:54:45,545 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6 2023-02-13 16:54:45,548 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b 2023-02-13 16:54:45,551 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 2023-02-13 16:54:45,554 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146 2023-02-13 16:54:45,557 DEBUG [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703 2023-02-13 16:54:45,557 INFO [StoreOpener-1ad9073aa1705b085f9e78eda1933547-1] regionserver.HStore(310): Store=1ad9073aa1705b085f9e78eda1933547/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:45,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:45,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:45,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.splits 2023-02-13 16:54:45,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:45,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:45,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1ad9073aa1705b085f9e78eda1933547; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61979990, jitterRate=-0.0764261782169342}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:45,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1ad9073aa1705b085f9e78eda1933547: Running coprocessor pre-open hook at 1676307285503Writing region info on filesystem at 1676307285503Initializing all the Stores at 1676307285504 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307285504Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307285504Cleaning up temporary data from old regions at 1676307285560 (+56 ms)Cleaning up detritus from prior splits at 1676307285560Running coprocessor post-open hooks at 1676307285563 (+3 ms)Region opened successfully at 1676307285563 2023-02-13 16:54:45,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547., pid=48, masterSystemTime=1676307285499 2023-02-13 16:54:45,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:45,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:54:45,565 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:45,565 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:45,566 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 16:54:45,566 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 16:54:45,566 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1912): 1ad9073aa1705b085f9e78eda1933547/cf1 is initiating minor compaction (all files) 2023-02-13 16:54:45,566 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1912): 1ad9073aa1705b085f9e78eda1933547/cf2 is initiating minor compaction (all files) 2023-02-13 16:54:45,566 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1ad9073aa1705b085f9e78eda1933547/cf1 in testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:45,566 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1ad9073aa1705b085f9e78eda1933547/cf2 in testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:45,567 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp, totalSize=34.9 K 2023-02-13 16:54:45,567 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp, totalSize=34.9 K 2023-02-13 16:54:45,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:45,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:45,567 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting b357314c7faf4fc1ab1e900060fae146, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307278198 2023-02-13 16:54:45,567 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 36071240f15943f49df2b7a68151e3dd, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307278198 2023-02-13 16:54:45,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:45,568 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 91a71c6a6e4a4d02a32dd7735f7f2958, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307278384 2023-02-13 16:54:45,568 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307285567"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307285567"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307285567"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307285567"}]},"ts":"1676307285567"} 2023-02-13 16:54:45,568 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting e4a7f2594e7f4149b2dd18583cae833d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307278384 2023-02-13 16:54:45,568 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 27c693f57584430d99b495942e0f67e6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307278583 2023-02-13 16:54:45,568 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 1149efdf750b4b69bd3108b446b27289, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307278583 2023-02-13 16:54:45,568 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting cb417272edfb4d9cb5cea06ff823a703, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307278773 2023-02-13 16:54:45,569 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 0047f9f32cfc4b51b97de82713fe1c68, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307278773 2023-02-13 16:54:45,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-13 16:54:45,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 in 222 msec 2023-02-13 16:54:45,572 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 1cbfaadbaf984da4a9f9c7acbc5e608f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307278950 2023-02-13 16:54:45,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN in 1.5460 sec 2023-02-13 16:54:45,572 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting f30e19700bca4b90a11eebe72969f448, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307278950 2023-02-13 16:54:45,572 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 142664c8dc1342f0ac4fa30ff8081f88, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307279136 2023-02-13 16:54:45,572 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting d381b11f63164380bbc8ad286993df9f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307279136 2023-02-13 16:54:45,573 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 5505df4447a04cc2bb74661b98bcdc4b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307279319 2023-02-13 16:54:45,573 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 99f1322d133349caa900766fb998ad97, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307279319 2023-02-13 16:54:45,599 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] throttle.PressureAwareThroughputController(145): 1ad9073aa1705b085f9e78eda1933547#cf2#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:45,603 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1ad9073aa1705b085f9e78eda1933547#cf1#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:45,621 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf2/c2f56a1289f1404db81f8be3eaf14962 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/c2f56a1289f1404db81f8be3eaf14962 2023-02-13 16:54:45,623 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/.tmp/cf1/8407811a51c049129d3ded2e6270b3b5 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/8407811a51c049129d3ded2e6270b3b5 2023-02-13 16:54:45,626 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1ad9073aa1705b085f9e78eda1933547/cf2 of 1ad9073aa1705b085f9e78eda1933547 into c2f56a1289f1404db81f8be3eaf14962(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:45,626 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1ad9073aa1705b085f9e78eda1933547: 2023-02-13 16:54:45,626 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547., storeName=1ad9073aa1705b085f9e78eda1933547/cf2, priority=9, startTime=1676307285565; duration=0sec 2023-02-13 16:54:45,627 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:45,629 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1ad9073aa1705b085f9e78eda1933547/cf1 of 1ad9073aa1705b085f9e78eda1933547 into 8407811a51c049129d3ded2e6270b3b5(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:45,629 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1ad9073aa1705b085f9e78eda1933547: 2023-02-13 16:54:45,629 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547., storeName=1ad9073aa1705b085f9e78eda1933547/cf1, priority=9, startTime=1676307285564; duration=0sec 2023-02-13 16:54:45,629 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:45,922 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-13 16:54:45,924 DEBUG [Listener at localhost/38547] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-13 16:54:45,926 INFO [Listener at localhost/38547] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 16:54:45,928 INFO [Listener at localhost/38547] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-13 16:54:45,928 INFO [Listener at localhost/38547] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 16:54:45,928 INFO [Listener at localhost/38547] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-13 16:54:45,942 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:45,944 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-13 16:54:45,944 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-13 16:54:45,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:45,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 16:54:45,948 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307285948"}]},"ts":"1676307285948"} 2023-02-13 16:54:45,949 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 16:54:45,951 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-13 16:54:45,951 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN}] 2023-02-13 16:54:45,952 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN 2023-02-13 16:54:45,952 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:45,952 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307285952"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307285952"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307285952"}]},"ts":"1676307285952"} 2023-02-13 16:54:45,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:46,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 16:54:46,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:46,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1ad9073aa1705b085f9e78eda1933547, disabling compactions & flushes 2023-02-13 16:54:46,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:46,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:46,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. after waiting 0 ms 2023-02-13 16:54:46,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:46,106 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97] to archive 2023-02-13 16:54:46,108 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:46,110 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/36071240f15943f49df2b7a68151e3dd 2023-02-13 16:54:46,111 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/e4a7f2594e7f4149b2dd18583cae833d 2023-02-13 16:54:46,112 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/1149efdf750b4b69bd3108b446b27289 2023-02-13 16:54:46,113 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/0047f9f32cfc4b51b97de82713fe1c68 2023-02-13 16:54:46,115 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/f30e19700bca4b90a11eebe72969f448 2023-02-13 16:54:46,116 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/d381b11f63164380bbc8ad286993df9f 2023-02-13 16:54:46,117 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/99f1322d133349caa900766fb998ad97 2023-02-13 16:54:46,122 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b] to archive 2023-02-13 16:54:46,123 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:46,124 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/b357314c7faf4fc1ab1e900060fae146 2023-02-13 16:54:46,125 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/91a71c6a6e4a4d02a32dd7735f7f2958 2023-02-13 16:54:46,126 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/27c693f57584430d99b495942e0f67e6 2023-02-13 16:54:46,127 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/cb417272edfb4d9cb5cea06ff823a703 2023-02-13 16:54:46,129 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/1cbfaadbaf984da4a9f9c7acbc5e608f 2023-02-13 16:54:46,129 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/142664c8dc1342f0ac4fa30ff8081f88 2023-02-13 16:54:46,130 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/5505df4447a04cc2bb74661b98bcdc4b 2023-02-13 16:54:46,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:54:46,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:46,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547. 2023-02-13 16:54:46,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1ad9073aa1705b085f9e78eda1933547: Waiting for close lock at 1676307286105Running coprocessor pre-close hooks at 1676307286105Disabling compacts and flushes for region at 1676307286105Disabling writes for close at 1676307286106 (+1 ms)Writing region close event to WAL at 1676307286131 (+25 ms)Running coprocessor post-close hooks at 1676307286134 (+3 ms)Closed at 1676307286135 (+1 ms) 2023-02-13 16:54:46,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:46,136 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=1ad9073aa1705b085f9e78eda1933547, regionState=CLOSED 2023-02-13 16:54:46,137 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307286136"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307286136"}]},"ts":"1676307286136"} 2023-02-13 16:54:46,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 16:54:46,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 1ad9073aa1705b085f9e78eda1933547, server=jenkins-hbase4.apache.org,34717,1676307263076 in 184 msec 2023-02-13 16:54:46,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 16:54:46,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, UNASSIGN in 189 msec 2023-02-13 16:54:46,142 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307286142"}]},"ts":"1676307286142"} 2023-02-13 16:54:46,143 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 16:54:46,144 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-13 16:54:46,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 200 msec 2023-02-13 16:54:46,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 16:54:46,250 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-13 16:54:46,251 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-13 16:54:46,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:46,254 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:46,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 16:54:46,255 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:46,259 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:46,261 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/recovered.edits] 2023-02-13 16:54:46,264 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/8407811a51c049129d3ded2e6270b3b5 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf1/8407811a51c049129d3ded2e6270b3b5 2023-02-13 16:54:46,266 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/c2f56a1289f1404db81f8be3eaf14962 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/cf2/c2f56a1289f1404db81f8be3eaf14962 2023-02-13 16:54:46,269 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547/recovered.edits/90.seqid 2023-02-13 16:54:46,270 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackAndDoubleExecution/1ad9073aa1705b085f9e78eda1933547 2023-02-13 16:54:46,270 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 16:54:46,273 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:46,275 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-13 16:54:46,276 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-13 16:54:46,277 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:46,277 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-13 16:54:46,277 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307286277"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:46,279 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 16:54:46,279 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1ad9073aa1705b085f9e78eda1933547, NAME => 'testRollbackAndDoubleExecution,,1676307277694.1ad9073aa1705b085f9e78eda1933547.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 16:54:46,279 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-13 16:54:46,279 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307286279"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:46,281 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-13 16:54:46,282 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:54:46,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 31 msec 2023-02-13 16:54:46,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 16:54:46,356 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-13 16:54:46,370 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=396 (was 400), OpenFileDescriptor=864 (was 857) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 250), ProcessCount=168 (was 168), AvailableMemoryMB=1333 (was 1353) 2023-02-13 16:54:46,384 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=396, OpenFileDescriptor=864, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=168, AvailableMemoryMB=1333 2023-02-13 16:54:46,384 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:46,388 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:54:46,389 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:54:46,390 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:54:46,391 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:46,392 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:46,393 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,394 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1 empty. 2023-02-13 16:54:46,394 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,394 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 16:54:46,404 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:46,406 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9a580bc12d3425731fe759637781ddc1, NAME => 'testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:54:46,419 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:46,419 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 9a580bc12d3425731fe759637781ddc1, disabling compactions & flushes 2023-02-13 16:54:46,419 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:46,419 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:46,419 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. after waiting 0 ms 2023-02-13 16:54:46,419 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:46,419 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:46,419 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 9a580bc12d3425731fe759637781ddc1: Waiting for close lock at 1676307286419Disabling compacts and flushes for region at 1676307286419Disabling writes for close at 1676307286419Writing region close event to WAL at 1676307286419Closed at 1676307286419 2023-02-13 16:54:46,421 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:46,422 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307286422"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307286422"}]},"ts":"1676307286422"} 2023-02-13 16:54:46,423 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:46,424 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:46,424 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307286424"}]},"ts":"1676307286424"} 2023-02-13 16:54:46,425 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-13 16:54:46,428 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:54:46,429 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:54:46,429 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:54:46,429 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:54:46,429 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:54:46,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9a580bc12d3425731fe759637781ddc1, ASSIGN}] 2023-02-13 16:54:46,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9a580bc12d3425731fe759637781ddc1, ASSIGN 2023-02-13 16:54:46,431 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9a580bc12d3425731fe759637781ddc1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:46,581 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:54:46,583 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=9a580bc12d3425731fe759637781ddc1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:46,583 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307286583"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307286583"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307286583"}]},"ts":"1676307286583"} 2023-02-13 16:54:46,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 9a580bc12d3425731fe759637781ddc1, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:46,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:46,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9a580bc12d3425731fe759637781ddc1, NAME => 'testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:46,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:46,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:46,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,741 INFO [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,743 DEBUG [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/cf1 2023-02-13 16:54:46,743 DEBUG [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/cf1 2023-02-13 16:54:46,744 INFO [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a580bc12d3425731fe759637781ddc1 columnFamilyName cf1 2023-02-13 16:54:46,745 INFO [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] regionserver.HStore(310): Store=9a580bc12d3425731fe759637781ddc1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:46,745 INFO [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,746 DEBUG [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/cf2 2023-02-13 16:54:46,746 DEBUG [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/cf2 2023-02-13 16:54:46,747 INFO [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a580bc12d3425731fe759637781ddc1 columnFamilyName cf2 2023-02-13 16:54:46,747 INFO [StoreOpener-9a580bc12d3425731fe759637781ddc1-1] regionserver.HStore(310): Store=9a580bc12d3425731fe759637781ddc1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:46,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:46,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:46,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:46,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9a580bc12d3425731fe759637781ddc1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74869260, jitterRate=0.11563891172409058}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:46,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9a580bc12d3425731fe759637781ddc1: Running coprocessor pre-open hook at 1676307286740Writing region info on filesystem at 1676307286740Initializing all the Stores at 1676307286741 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307286741Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307286741Cleaning up temporary data from old regions at 1676307286749 (+8 ms)Cleaning up detritus from prior splits at 1676307286749Running coprocessor post-open hooks at 1676307286754 (+5 ms)Region opened successfully at 1676307286754 2023-02-13 16:54:46,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1., pid=55, masterSystemTime=1676307286736 2023-02-13 16:54:46,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:46,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:46,757 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=9a580bc12d3425731fe759637781ddc1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:46,757 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307286757"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307286757"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307286757"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307286757"}]},"ts":"1676307286757"} 2023-02-13 16:54:46,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-13 16:54:46,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 9a580bc12d3425731fe759637781ddc1, server=jenkins-hbase4.apache.org,34717,1676307263076 in 174 msec 2023-02-13 16:54:46,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-13 16:54:46,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9a580bc12d3425731fe759637781ddc1, ASSIGN in 331 msec 2023-02-13 16:54:46,763 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:46,763 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307286763"}]},"ts":"1676307286763"} 2023-02-13 16:54:46,764 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-13 16:54:46,767 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:46,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 378 msec 2023-02-13 16:54:46,891 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:46,892 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:46,893 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:46,893 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:46,893 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:46,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9a580bc12d3425731fe759637781ddc1, UNASSIGN}] 2023-02-13 16:54:46,896 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9a580bc12d3425731fe759637781ddc1, UNASSIGN 2023-02-13 16:54:46,896 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=9a580bc12d3425731fe759637781ddc1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:46,897 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307286896"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307286896"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307286896"}]},"ts":"1676307286896"} 2023-02-13 16:54:46,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 9a580bc12d3425731fe759637781ddc1, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:47,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:47,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9a580bc12d3425731fe759637781ddc1, disabling compactions & flushes 2023-02-13 16:54:47,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:47,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:47,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. after waiting 0 ms 2023-02-13 16:54:47,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:47,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 16:54:47,055 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:47,055 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1. 2023-02-13 16:54:47,055 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9a580bc12d3425731fe759637781ddc1: Waiting for close lock at 1676307287050Running coprocessor pre-close hooks at 1676307287050Disabling compacts and flushes for region at 1676307287050Disabling writes for close at 1676307287051 (+1 ms)Writing region close event to WAL at 1676307287051Running coprocessor post-close hooks at 1676307287055 (+4 ms)Closed at 1676307287055 2023-02-13 16:54:47,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:47,057 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=9a580bc12d3425731fe759637781ddc1, regionState=CLOSED 2023-02-13 16:54:47,057 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307287057"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307287057"}]},"ts":"1676307287057"} 2023-02-13 16:54:47,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-13 16:54:47,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 9a580bc12d3425731fe759637781ddc1, server=jenkins-hbase4.apache.org,34717,1676307263076 in 161 msec 2023-02-13 16:54:47,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 16:54:47,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=9a580bc12d3425731fe759637781ddc1, UNASSIGN in 171 msec 2023-02-13 16:54:47,130 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 16:54:47,133 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 16:54:47,135 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307287135"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676307287135"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676307287135"}]},"ts":"1676307287135"} 2023-02-13 16:54:47,135 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287135"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307287135"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307287135"}]},"ts":"1676307287135"} 2023-02-13 16:54:47,135 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287135"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307287135"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307287135"}]},"ts":"1676307287135"} 2023-02-13 16:54:47,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8261b510c81f764ac5b4f1f2b9db29aa, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac409e0d07945beddac77394f7974c3c, ASSIGN}] 2023-02-13 16:54:47,140 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac409e0d07945beddac77394f7974c3c, ASSIGN 2023-02-13 16:54:47,140 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac409e0d07945beddac77394f7974c3c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:47,141 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8261b510c81f764ac5b4f1f2b9db29aa, ASSIGN 2023-02-13 16:54:47,142 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8261b510c81f764ac5b4f1f2b9db29aa, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:47,291 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 16:54:47,292 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=8261b510c81f764ac5b4f1f2b9db29aa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:47,293 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287292"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307287292"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307287292"}]},"ts":"1676307287292"} 2023-02-13 16:54:47,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 8261b510c81f764ac5b4f1f2b9db29aa, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:47,297 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=ac409e0d07945beddac77394f7974c3c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:47,297 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287297"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307287297"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307287297"}]},"ts":"1676307287297"} 2023-02-13 16:54:47,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure ac409e0d07945beddac77394f7974c3c, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:47,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa. 2023-02-13 16:54:47,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8261b510c81f764ac5b4f1f2b9db29aa, NAME => 'testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 16:54:47,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:47,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:47,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,451 INFO [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,452 DEBUG [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/cf1 2023-02-13 16:54:47,452 DEBUG [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/cf1 2023-02-13 16:54:47,453 INFO [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8261b510c81f764ac5b4f1f2b9db29aa columnFamilyName cf1 2023-02-13 16:54:47,453 INFO [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] regionserver.HStore(310): Store=8261b510c81f764ac5b4f1f2b9db29aa/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:47,453 INFO [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,454 DEBUG [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/cf2 2023-02-13 16:54:47,454 DEBUG [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/cf2 2023-02-13 16:54:47,454 INFO [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8261b510c81f764ac5b4f1f2b9db29aa columnFamilyName cf2 2023-02-13 16:54:47,455 INFO [StoreOpener-8261b510c81f764ac5b4f1f2b9db29aa-1] regionserver.HStore(310): Store=8261b510c81f764ac5b4f1f2b9db29aa/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:47,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:47,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8261b510c81f764ac5b4f1f2b9db29aa; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61681544, jitterRate=-0.08087337017059326}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:47,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8261b510c81f764ac5b4f1f2b9db29aa: Running coprocessor pre-open hook at 1676307287450Writing region info on filesystem at 1676307287450Initializing all the Stores at 1676307287450Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307287451 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307287451Cleaning up temporary data from old regions at 1676307287457 (+6 ms)Cleaning up detritus from prior splits at 1676307287457Running coprocessor post-open hooks at 1676307287459 (+2 ms)Region opened successfully at 1676307287459 2023-02-13 16:54:47,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa., pid=61, masterSystemTime=1676307287446 2023-02-13 16:54:47,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa. 2023-02-13 16:54:47,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa. 2023-02-13 16:54:47,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c. 2023-02-13 16:54:47,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ac409e0d07945beddac77394f7974c3c, NAME => 'testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 16:54:47,462 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=8261b510c81f764ac5b4f1f2b9db29aa, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:47,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:47,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287462"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307287462"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307287462"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307287462"}]},"ts":"1676307287462"} 2023-02-13 16:54:47,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:47,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,464 INFO [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,465 DEBUG [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/cf1 2023-02-13 16:54:47,465 DEBUG [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/cf1 2023-02-13 16:54:47,466 INFO [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ac409e0d07945beddac77394f7974c3c columnFamilyName cf1 2023-02-13 16:54:47,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-13 16:54:47,466 INFO [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] regionserver.HStore(310): Store=ac409e0d07945beddac77394f7974c3c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:47,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 8261b510c81f764ac5b4f1f2b9db29aa, server=jenkins-hbase4.apache.org,34717,1676307263076 in 170 msec 2023-02-13 16:54:47,466 INFO [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8261b510c81f764ac5b4f1f2b9db29aa, ASSIGN in 327 msec 2023-02-13 16:54:47,468 DEBUG [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/cf2 2023-02-13 16:54:47,468 DEBUG [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/cf2 2023-02-13 16:54:47,468 INFO [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ac409e0d07945beddac77394f7974c3c columnFamilyName cf2 2023-02-13 16:54:47,469 INFO [StoreOpener-ac409e0d07945beddac77394f7974c3c-1] regionserver.HStore(310): Store=ac409e0d07945beddac77394f7974c3c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:47,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:47,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ac409e0d07945beddac77394f7974c3c; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71885646, jitterRate=0.07117959856987}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:47,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ac409e0d07945beddac77394f7974c3c: Running coprocessor pre-open hook at 1676307287463Writing region info on filesystem at 1676307287463Initializing all the Stores at 1676307287464 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307287464Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307287464Cleaning up temporary data from old regions at 1676307287471 (+7 ms)Cleaning up detritus from prior splits at 1676307287471Running coprocessor post-open hooks at 1676307287474 (+3 ms)Region opened successfully at 1676307287474 2023-02-13 16:54:47,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c., pid=62, masterSystemTime=1676307287446 2023-02-13 16:54:47,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c. 2023-02-13 16:54:47,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c. 2023-02-13 16:54:47,477 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=ac409e0d07945beddac77394f7974c3c, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:47,477 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287477"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307287477"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307287477"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307287477"}]},"ts":"1676307287477"} 2023-02-13 16:54:47,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-13 16:54:47,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure ac409e0d07945beddac77394f7974c3c, server=jenkins-hbase4.apache.org,34717,1676307263076 in 180 msec 2023-02-13 16:54:47,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-02-13 16:54:47,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac409e0d07945beddac77394f7974c3c, ASSIGN in 341 msec 2023-02-13 16:54:47,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c in 590 msec 2023-02-13 16:54:47,647 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 16:54:47,650 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:47,651 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,652 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 16:54:47,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307287655"}]},"ts":"1676307287655"} 2023-02-13 16:54:47,656 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-13 16:54:47,659 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-13 16:54:47,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8261b510c81f764ac5b4f1f2b9db29aa, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac409e0d07945beddac77394f7974c3c, UNASSIGN}] 2023-02-13 16:54:47,660 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac409e0d07945beddac77394f7974c3c, UNASSIGN 2023-02-13 16:54:47,661 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=ac409e0d07945beddac77394f7974c3c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:47,661 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287661"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307287661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307287661"}]},"ts":"1676307287661"} 2023-02-13 16:54:47,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure ac409e0d07945beddac77394f7974c3c, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:47,664 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8261b510c81f764ac5b4f1f2b9db29aa, UNASSIGN 2023-02-13 16:54:47,665 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=8261b510c81f764ac5b4f1f2b9db29aa, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:47,665 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287665"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307287665"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307287665"}]},"ts":"1676307287665"} 2023-02-13 16:54:47,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 8261b510c81f764ac5b4f1f2b9db29aa, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:47,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 16:54:47,814 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8261b510c81f764ac5b4f1f2b9db29aa, disabling compactions & flushes 2023-02-13 16:54:47,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa. 2023-02-13 16:54:47,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa. 2023-02-13 16:54:47,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa. after waiting 0 ms 2023-02-13 16:54:47,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa. 2023-02-13 16:54:47,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 16:54:47,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:47,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa. 2023-02-13 16:54:47,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8261b510c81f764ac5b4f1f2b9db29aa: Waiting for close lock at 1676307287814Running coprocessor pre-close hooks at 1676307287814Disabling compacts and flushes for region at 1676307287814Disabling writes for close at 1676307287815 (+1 ms)Writing region close event to WAL at 1676307287815Running coprocessor post-close hooks at 1676307287819 (+4 ms)Closed at 1676307287819 2023-02-13 16:54:47,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ac409e0d07945beddac77394f7974c3c, disabling compactions & flushes 2023-02-13 16:54:47,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c. 2023-02-13 16:54:47,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c. 2023-02-13 16:54:47,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c. after waiting 0 ms 2023-02-13 16:54:47,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c. 2023-02-13 16:54:47,822 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=8261b510c81f764ac5b4f1f2b9db29aa, regionState=CLOSED 2023-02-13 16:54:47,822 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287822"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307287822"}]},"ts":"1676307287822"} 2023-02-13 16:54:47,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 16:54:47,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-13 16:54:47,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 8261b510c81f764ac5b4f1f2b9db29aa, server=jenkins-hbase4.apache.org,34717,1676307263076 in 157 msec 2023-02-13 16:54:47,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:47,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c. 2023-02-13 16:54:47,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ac409e0d07945beddac77394f7974c3c: Waiting for close lock at 1676307287821Running coprocessor pre-close hooks at 1676307287821Disabling compacts and flushes for region at 1676307287821Disabling writes for close at 1676307287822 (+1 ms)Writing region close event to WAL at 1676307287822Running coprocessor post-close hooks at 1676307287825 (+3 ms)Closed at 1676307287825 2023-02-13 16:54:47,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8261b510c81f764ac5b4f1f2b9db29aa, UNASSIGN in 166 msec 2023-02-13 16:54:47,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,827 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=ac409e0d07945beddac77394f7974c3c, regionState=CLOSED 2023-02-13 16:54:47,827 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307287827"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307287827"}]},"ts":"1676307287827"} 2023-02-13 16:54:47,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 16:54:47,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure ac409e0d07945beddac77394f7974c3c, server=jenkins-hbase4.apache.org,34717,1676307263076 in 166 msec 2023-02-13 16:54:47,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-13 16:54:47,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac409e0d07945beddac77394f7974c3c, UNASSIGN in 171 msec 2023-02-13 16:54:47,831 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307287831"}]},"ts":"1676307287831"} 2023-02-13 16:54:47,833 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-13 16:54:47,834 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-13 16:54:47,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 182 msec 2023-02-13 16:54:47,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 16:54:47,957 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-13 16:54:47,957 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,960 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 16:54:47,960 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,965 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:47,965 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,965 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,968 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/recovered.edits] 2023-02-13 16:54:47,968 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/recovered.edits] 2023-02-13 16:54:47,968 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/recovered.edits] 2023-02-13 16:54:47,975 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/recovered.edits/7.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa/recovered.edits/7.seqid 2023-02-13 16:54:47,975 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/recovered.edits/7.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c/recovered.edits/7.seqid 2023-02-13 16:54:47,976 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/recovered.edits/4.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1/recovered.edits/4.seqid 2023-02-13 16:54:47,976 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/8261b510c81f764ac5b4f1f2b9db29aa 2023-02-13 16:54:47,976 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/ac409e0d07945beddac77394f7974c3c 2023-02-13 16:54:47,976 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionNoStoreFile/9a580bc12d3425731fe759637781ddc1 2023-02-13 16:54:47,976 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 16:54:47,978 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,980 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-13 16:54:47,982 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-13 16:54:47,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-13 16:54:47,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307287983"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:47,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307287983"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:47,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307287983"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:47,984 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 16:54:47,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 9a580bc12d3425731fe759637781ddc1, NAME => 'testSplitTableRegionNoStoreFile,,1676307286389.9a580bc12d3425731fe759637781ddc1.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 8261b510c81f764ac5b4f1f2b9db29aa, NAME => 'testSplitTableRegionNoStoreFile,,1676307286891.8261b510c81f764ac5b4f1f2b9db29aa.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => ac409e0d07945beddac77394f7974c3c, NAME => 'testSplitTableRegionNoStoreFile,41,1676307286891.ac409e0d07945beddac77394f7974c3c.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 16:54:47,984 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-13 16:54:47,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307287984"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:47,986 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-13 16:54:47,987 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:54:47,988 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 30 msec 2023-02-13 16:54:48,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 16:54:48,061 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-13 16:54:48,075 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=400 (was 396) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1259053484_15 at /127.0.0.1:51648 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1259053484_15 at /127.0.0.1:52982 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=864 (was 864), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 229), ProcessCount=168 (was 168), AvailableMemoryMB=1327 (was 1333) 2023-02-13 16:54:48,089 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=400, OpenFileDescriptor=864, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=168, AvailableMemoryMB=1327 2023-02-13 16:54:48,089 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:48,091 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:54:48,092 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:54:48,093 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,094 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:48,095 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:48,096 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,097 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def empty. 2023-02-13 16:54:48,097 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,097 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 16:54:48,111 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:48,112 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7ad67c2797888267addece4f08ec7def, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:54:48,122 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:48,122 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 7ad67c2797888267addece4f08ec7def, disabling compactions & flushes 2023-02-13 16:54:48,122 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,122 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,122 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. after waiting 0 ms 2023-02-13 16:54:48,123 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,123 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,123 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 7ad67c2797888267addece4f08ec7def: Waiting for close lock at 1676307288122Disabling compacts and flushes for region at 1676307288122Disabling writes for close at 1676307288123 (+1 ms)Writing region close event to WAL at 1676307288123Closed at 1676307288123 2023-02-13 16:54:48,125 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:48,125 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676307288125"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307288125"}]},"ts":"1676307288125"} 2023-02-13 16:54:48,127 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:48,127 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:48,128 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307288128"}]},"ts":"1676307288128"} 2023-02-13 16:54:48,129 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-13 16:54:48,132 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:54:48,133 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:54:48,133 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:54:48,133 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:54:48,133 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:54:48,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7ad67c2797888267addece4f08ec7def, ASSIGN}] 2023-02-13 16:54:48,134 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7ad67c2797888267addece4f08ec7def, ASSIGN 2023-02-13 16:54:48,135 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7ad67c2797888267addece4f08ec7def, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:48,285 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:54:48,292 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=7ad67c2797888267addece4f08ec7def, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,292 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676307288292"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307288292"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307288292"}]},"ts":"1676307288292"} 2023-02-13 16:54:48,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 7ad67c2797888267addece4f08ec7def, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:48,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7ad67c2797888267addece4f08ec7def, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:48,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:48,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:48,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,453 INFO [StoreOpener-7ad67c2797888267addece4f08ec7def-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,454 DEBUG [StoreOpener-7ad67c2797888267addece4f08ec7def-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1 2023-02-13 16:54:48,454 DEBUG [StoreOpener-7ad67c2797888267addece4f08ec7def-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1 2023-02-13 16:54:48,454 INFO [StoreOpener-7ad67c2797888267addece4f08ec7def-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ad67c2797888267addece4f08ec7def columnFamilyName cf1 2023-02-13 16:54:48,455 INFO [StoreOpener-7ad67c2797888267addece4f08ec7def-1] regionserver.HStore(310): Store=7ad67c2797888267addece4f08ec7def/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:48,455 INFO [StoreOpener-7ad67c2797888267addece4f08ec7def-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,456 DEBUG [StoreOpener-7ad67c2797888267addece4f08ec7def-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2 2023-02-13 16:54:48,456 DEBUG [StoreOpener-7ad67c2797888267addece4f08ec7def-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2 2023-02-13 16:54:48,456 INFO [StoreOpener-7ad67c2797888267addece4f08ec7def-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ad67c2797888267addece4f08ec7def columnFamilyName cf2 2023-02-13 16:54:48,457 INFO [StoreOpener-7ad67c2797888267addece4f08ec7def-1] regionserver.HStore(310): Store=7ad67c2797888267addece4f08ec7def/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:48,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:48,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:48,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:48,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7ad67c2797888267addece4f08ec7def; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65157922, jitterRate=-0.029071301221847534}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:48,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7ad67c2797888267addece4f08ec7def: Running coprocessor pre-open hook at 1676307288452Writing region info on filesystem at 1676307288452Initializing all the Stores at 1676307288452Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307288452Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307288452Cleaning up temporary data from old regions at 1676307288458 (+6 ms)Cleaning up detritus from prior splits at 1676307288459 (+1 ms)Running coprocessor post-open hooks at 1676307288464 (+5 ms)Region opened successfully at 1676307288464 2023-02-13 16:54:48,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def., pid=71, masterSystemTime=1676307288448 2023-02-13 16:54:48,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,469 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=7ad67c2797888267addece4f08ec7def, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,469 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676307288469"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307288469"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307288469"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307288469"}]},"ts":"1676307288469"} 2023-02-13 16:54:48,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-13 16:54:48,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 7ad67c2797888267addece4f08ec7def, server=jenkins-hbase4.apache.org,34717,1676307263076 in 174 msec 2023-02-13 16:54:48,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-13 16:54:48,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7ad67c2797888267addece4f08ec7def, ASSIGN in 338 msec 2023-02-13 16:54:48,474 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:48,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307288474"}]},"ts":"1676307288474"} 2023-02-13 16:54:48,475 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-13 16:54:48,477 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:48,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 385 msec 2023-02-13 16:54:48,601 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:48,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,602 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:48,602 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,602 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 16:54:48,602 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 16:54:48,603 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,603 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,604 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,604 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,604 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,604 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,604 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,604 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:48,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,606 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:48,606 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:48,606 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,606 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. started... 2023-02-13 16:54:48,607 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 16:54:48,608 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,609 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,609 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:48,609 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,610 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,610 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:48,622 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/928a7b3e745d4bf8b0152f6117c6bc47 2023-02-13 16:54:48,638 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/600ac40c07764deeaee534f86d1434d5 2023-02-13 16:54:48,642 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/928a7b3e745d4bf8b0152f6117c6bc47 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47 2023-02-13 16:54:48,647 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:54:48,647 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/600ac40c07764deeaee534f86d1434d5 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5 2023-02-13 16:54:48,652 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:54:48,652 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 45ms, sequenceid=6, compaction requested=false 2023-02-13 16:54:48,653 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:48,653 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ad67c2797888267addece4f08ec7def: 2023-02-13 16:54:48,653 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,653 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:48,653 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:48,653 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,653 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:48,653 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,655 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:48,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:48,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:48,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 16:54:48,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b62cc09[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:48,661 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:48,661 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,662 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,662 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,662 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,663 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,663 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:48,663 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,663 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:48,663 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:48,663 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:48,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,663 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,663 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:48,663 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:48,663 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:48,663 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:48,665 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,665 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:48,665 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:48,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:48,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:48,665 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:48,666 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:48,666 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:48,666 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:48,667 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:48,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:48,667 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:48,667 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:48,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:48,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:48,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:48,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:48,670 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 16:54:48,670 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:48,670 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:48,670 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,670 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:48,672 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:48,672 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,672 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:48,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,674 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:48,675 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,675 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:48,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,677 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,677 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,677 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,684 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:48,684 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:48,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,685 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,685 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:48,685 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:48,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,684 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:48,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,685 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,685 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,685 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,685 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,685 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,785 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:48,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:48,798 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:48,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:48,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,800 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:48,800 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,800 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 16:54:48,800 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 16:54:48,801 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,801 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,802 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,802 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,802 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,802 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,802 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,802 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:48,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:48,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:48,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:48,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:48,804 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:48,805 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:48,805 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,805 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. started... 2023-02-13 16:54:48,805 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:48,806 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:48,820 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/edc584f92e74472899a980e41ef24c35 2023-02-13 16:54:48,837 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/77139cb766334e39891ea13c6b6b987e 2023-02-13 16:54:48,841 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/edc584f92e74472899a980e41ef24c35 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35 2023-02-13 16:54:48,846 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:54:48,847 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/77139cb766334e39891ea13c6b6b987e as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e 2023-02-13 16:54:48,850 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:54:48,851 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 46ms, sequenceid=19, compaction requested=false 2023-02-13 16:54:48,851 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ad67c2797888267addece4f08ec7def: 2023-02-13 16:54:48,851 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,851 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:48,851 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:48,851 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,851 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:48,851 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:48,854 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,854 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:48,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:48,854 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,854 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:48,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:48,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:48,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:48,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 16:54:48,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50093aa4[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:48,856 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:48,857 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:48,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:48,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:48,859 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:48,859 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,859 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:48,859 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:48,861 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:48,861 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:48,861 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:48,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:48,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:48,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:48,865 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:48,865 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 16:54:48,865 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:48,865 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:48,865 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,865 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:48,866 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,866 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,866 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:48,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:48,866 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,866 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,866 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:48,867 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,867 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,868 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,868 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,870 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,870 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,871 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,873 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,873 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,873 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,874 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:48,873 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,874 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:48,874 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:48,874 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,873 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,874 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:48,874 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:48,874 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:48,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,874 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,874 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,874 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,874 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,875 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:48,875 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,875 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,875 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:48,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:48,875 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,875 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,974 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:48,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:48,990 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:48,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:48,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,992 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:48,992 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,992 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 16:54:48,992 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 16:54:48,992 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,992 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,993 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,993 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,993 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,993 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:48,994 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:48,994 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:48,995 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:48,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:48,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,996 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:48,996 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:48,996 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. started... 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:48,996 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:48,996 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:48,999 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,999 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,999 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,999 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:48,999 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:48,999 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,012 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/e0bad3d69ee84f9a81d996df11eb6f80 2023-02-13 16:54:49,027 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/4941ac96b48147d0bc56e9194304db34 2023-02-13 16:54:49,032 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/e0bad3d69ee84f9a81d996df11eb6f80 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80 2023-02-13 16:54:49,036 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:54:49,037 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/4941ac96b48147d0bc56e9194304db34 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34 2023-02-13 16:54:49,041 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:54:49,042 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 46ms, sequenceid=32, compaction requested=true 2023-02-13 16:54:49,043 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ad67c2797888267addece4f08ec7def: 2023-02-13 16:54:49,043 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,043 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:49,043 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:49,043 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,043 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,043 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,049 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,049 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,049 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,049 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 16:54:49,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@605d3af3[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:49,051 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:49,051 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,052 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,052 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,052 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,052 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,052 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,052 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,053 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:49,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,055 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,056 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,056 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,056 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,056 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:49,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:49,059 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 16:54:49,059 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:49,059 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:49,059 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,059 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:49,060 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,060 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,060 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,060 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,060 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,061 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,061 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,061 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,062 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,062 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,064 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,064 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,065 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,068 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,068 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,068 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,068 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:49,068 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:49,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,068 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,068 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,068 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,069 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,069 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,069 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,168 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:49,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:49,189 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:49,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:49,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,191 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,191 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,191 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 16:54:49,191 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 16:54:49,192 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,192 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,194 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,194 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,194 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,194 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,194 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,194 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:49,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,196 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,197 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,197 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:49,197 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:49,197 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,197 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. started... 2023-02-13 16:54:49,197 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:49,198 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,199 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,199 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,199 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,199 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,199 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,214 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/940edeefad29403aafea01368c1978ba 2023-02-13 16:54:49,228 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/0b3ef2b5950546d993bd5f1c8e5fb350 2023-02-13 16:54:49,239 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/940edeefad29403aafea01368c1978ba as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/940edeefad29403aafea01368c1978ba 2023-02-13 16:54:49,244 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/940edeefad29403aafea01368c1978ba, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:54:49,245 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/0b3ef2b5950546d993bd5f1c8e5fb350 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0b3ef2b5950546d993bd5f1c8e5fb350 2023-02-13 16:54:49,250 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0b3ef2b5950546d993bd5f1c8e5fb350, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:54:49,251 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 54ms, sequenceid=45, compaction requested=true 2023-02-13 16:54:49,251 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ad67c2797888267addece4f08ec7def: 2023-02-13 16:54:49,251 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,251 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:49,251 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:49,251 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,251 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,251 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,252 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,252 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,253 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,253 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 16:54:49,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@273543e4[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:49,255 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:49,255 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,258 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,258 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:49,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,258 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,258 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,258 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:49,258 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:49,258 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,259 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,259 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:49,260 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,260 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,260 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,260 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,261 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,261 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,261 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:49,265 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:49,265 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 16:54:49,265 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:49,265 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:49,265 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,265 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:49,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,269 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,272 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,273 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,273 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,281 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,281 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,281 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,281 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:49,281 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,280 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,281 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,281 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,281 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:49,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,281 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,281 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:49,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,282 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,282 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,282 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,282 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,282 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,282 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,282 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,382 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:49,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:49,395 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:49,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:49,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,397 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,397 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,397 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 16:54:49,397 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 16:54:49,397 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,397 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,398 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,398 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,398 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,398 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,399 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,399 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:49,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,399 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,399 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,399 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,400 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,400 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,400 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,400 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,400 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,400 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,400 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,400 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,401 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,401 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,401 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,400 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:49,401 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:49,401 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,400 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,401 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,401 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,401 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. started... 2023-02-13 16:54:49,401 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:49,402 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,403 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,403 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,403 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,403 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,403 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,411 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/35dde213e7da43e3ab980e17ecab1352 2023-02-13 16:54:49,422 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/a77ffdb8e67f4c618f5c97b41dfcc713 2023-02-13 16:54:49,427 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/35dde213e7da43e3ab980e17ecab1352 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/35dde213e7da43e3ab980e17ecab1352 2023-02-13 16:54:49,431 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/35dde213e7da43e3ab980e17ecab1352, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:54:49,431 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/a77ffdb8e67f4c618f5c97b41dfcc713 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/a77ffdb8e67f4c618f5c97b41dfcc713 2023-02-13 16:54:49,435 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/a77ffdb8e67f4c618f5c97b41dfcc713, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:54:49,436 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 35ms, sequenceid=58, compaction requested=true 2023-02-13 16:54:49,436 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ad67c2797888267addece4f08ec7def: 2023-02-13 16:54:49,436 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,436 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:49,436 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:49,436 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,436 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,436 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,439 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,439 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,440 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,440 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 16:54:49,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3aaad18f[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:49,442 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:49,442 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,443 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,443 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,443 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,444 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,444 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,444 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:49,444 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,444 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,444 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:49,444 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,444 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:49,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,444 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,444 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,444 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:49,445 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,445 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,446 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,445 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,446 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,446 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,446 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,446 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,446 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,446 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:49,450 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:49,450 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 16:54:49,450 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:49,450 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:49,450 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,450 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:49,453 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,453 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,453 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,453 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,454 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,453 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,453 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,454 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,455 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,455 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,457 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,457 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,457 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,460 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,461 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:49,460 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,460 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,461 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,461 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:49,461 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,461 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,461 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:49,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,461 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,462 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,562 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:49,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:49,574 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:49,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:49,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,575 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,575 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,575 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 16:54:49,575 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 16:54:49,576 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,576 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,580 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,580 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,580 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,580 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,580 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,580 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:49,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,581 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,581 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,581 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:49,582 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:49,582 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,582 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. started... 2023-02-13 16:54:49,582 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:54:49,584 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,584 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,585 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,585 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,585 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,585 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,591 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/f9c336170025405e9269e32fc2a49fdc 2023-02-13 16:54:49,604 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/363c25063fa843e0b9fc36c7c2b909a2 2023-02-13 16:54:49,609 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/f9c336170025405e9269e32fc2a49fdc as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/f9c336170025405e9269e32fc2a49fdc 2023-02-13 16:54:49,612 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/f9c336170025405e9269e32fc2a49fdc, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:54:49,613 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/363c25063fa843e0b9fc36c7c2b909a2 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/363c25063fa843e0b9fc36c7c2b909a2 2023-02-13 16:54:49,616 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/363c25063fa843e0b9fc36c7c2b909a2, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:54:49,617 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 35ms, sequenceid=71, compaction requested=true 2023-02-13 16:54:49,617 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ad67c2797888267addece4f08ec7def: 2023-02-13 16:54:49,617 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,617 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:49,617 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:49,617 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,617 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,617 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,619 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,619 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,619 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,619 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 16:54:49,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@236f1064[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:49,621 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:49,621 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,622 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,622 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,622 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,623 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,623 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,623 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,623 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:49,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,623 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:49,623 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,623 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,623 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:49,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,623 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,623 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,623 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:49,626 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,626 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,626 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,626 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,626 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,626 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,626 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,626 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,626 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,626 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:49,630 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:49,630 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 16:54:49,630 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:49,630 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:49,630 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,630 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:49,632 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,632 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,633 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,633 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,635 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,635 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,635 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,637 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,637 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,637 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,638 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,638 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,637 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,638 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,638 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,638 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,638 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,638 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,638 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,638 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,638 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,639 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:49,639 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,639 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,639 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,639 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,639 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,639 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,639 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,739 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:49,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:49,752 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:49,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:49,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,754 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,754 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,754 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 16:54:49,754 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 16:54:49,755 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,755 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,757 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,757 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,757 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,757 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,757 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,757 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:49,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,759 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:49,759 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:49,759 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. started... 2023-02-13 16:54:49,760 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-13 16:54:49,761 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,761 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,761 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,761 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,761 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,761 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,770 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/caafec81ba6249bdb7e6faecf3bea1ce 2023-02-13 16:54:49,774 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for caafec81ba6249bdb7e6faecf3bea1ce 2023-02-13 16:54:49,782 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/7b05c187b729402ab0b3cf953e6b76d0 2023-02-13 16:54:49,785 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b05c187b729402ab0b3cf953e6b76d0 2023-02-13 16:54:49,785 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/caafec81ba6249bdb7e6faecf3bea1ce as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce 2023-02-13 16:54:49,788 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for caafec81ba6249bdb7e6faecf3bea1ce 2023-02-13 16:54:49,789 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 16:54:49,789 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/7b05c187b729402ab0b3cf953e6b76d0 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0 2023-02-13 16:54:49,792 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b05c187b729402ab0b3cf953e6b76d0 2023-02-13 16:54:49,793 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 16:54:49,793 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 34ms, sequenceid=83, compaction requested=true 2023-02-13 16:54:49,793 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ad67c2797888267addece4f08ec7def: 2023-02-13 16:54:49,793 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,794 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:49,794 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:49,794 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,794 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,794 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,795 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,795 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,796 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,796 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 16:54:49,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2aec3c58[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:49,797 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:49,797 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,799 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,799 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:49,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:49,801 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,801 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,801 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,804 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,804 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,804 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,804 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,804 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,804 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,805 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:49,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:49,807 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 16:54:49,807 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:49,807 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:49,807 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,807 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:49,809 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,809 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,809 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,809 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,809 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,810 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,810 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,811 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,812 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,812 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,816 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,816 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,816 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,817 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:49,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,817 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,817 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:49,817 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,817 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,917 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:49,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:49,925 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:49,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:49,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,926 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,926 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,926 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 16:54:49,926 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 16:54:49,927 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,927 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,928 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,928 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,928 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,928 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,929 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,929 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:49,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,930 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,930 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,930 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,930 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,930 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:49,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,930 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,930 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:49,930 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. started... 2023-02-13 16:54:49,931 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:49,931 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:49,931 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,931 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,931 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 16:54:49,932 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,932 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,932 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,933 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,933 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,933 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,939 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:49,943 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:49,950 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:49,953 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:49,954 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/2d6566efde434b7ababfd8a51409f018 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:49,957 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:49,957 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2d6566efde434b7ababfd8a51409f018, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 16:54:49,958 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/38b99d9751db46d38a4b76d3715644af as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:49,961 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:49,961 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/38b99d9751db46d38a4b76d3715644af, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 16:54:49,962 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 31ms, sequenceid=91, compaction requested=true 2023-02-13 16:54:49,962 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ad67c2797888267addece4f08ec7def: 2023-02-13 16:54:49,962 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:49,962 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:49,962 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:49,962 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,962 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:49,962 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:49,964 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,965 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,965 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,965 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 16:54:49,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27cb7082[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:49,967 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:49,967 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,968 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,968 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,968 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,968 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,968 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,968 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,968 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:49,968 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,968 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,969 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:49,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,969 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:49,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,969 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:49,969 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:49,969 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:49,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,970 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,970 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,970 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,970 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:49,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:49,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:49,974 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 16:54:49,974 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:49,974 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:49,974 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,974 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:49,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:49,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,977 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:49,977 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,978 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,979 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,979 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:49,980 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,985 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,985 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,985 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:49,985 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:49,985 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,985 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:49,985 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:49,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,985 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,985 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,985 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:49,985 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:49,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,985 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:49,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:49,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:49,986 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,086 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:50,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:50,094 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:50,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:50,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,096 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:50,096 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:50,096 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 16:54:50,096 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 16:54:50,096 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,096 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,097 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:50,097 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,097 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:50,097 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:50,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:50,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:50,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:50,098 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,098 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,098 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,098 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,098 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:50,099 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:50,099 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 16:54:50,099 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:50,099 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:50,099 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:50,100 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. started... 2023-02-13 16:54:50,100 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 16:54:50,100 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:50,100 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:50,100 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 16:54:50,103 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,104 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,104 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:50,104 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,104 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,104 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:50,108 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:50,112 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:50,121 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:50,124 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:50,124 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/b5fd8b19145d4baf8c1d11efee74effc as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:50,128 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:50,128 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/b5fd8b19145d4baf8c1d11efee74effc, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 16:54:50,129 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/0cfceef9bb1643c79da10fdb3eea35c0 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:50,133 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:50,133 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0cfceef9bb1643c79da10fdb3eea35c0, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 16:54:50,134 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 34ms, sequenceid=99, compaction requested=true 2023-02-13 16:54:50,134 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ad67c2797888267addece4f08ec7def: 2023-02-13 16:54:50,134 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:50,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:50,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:50,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 16:54:50,134 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 16:54:50,135 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,135 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:50,136 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,136 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:50,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 16:54:50,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b81fb5d[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:50,137 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 16:54:50,137 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,139 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:50,139 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:50,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,139 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,139 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:50,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,139 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:50,139 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 16:54:50,139 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:54:50,139 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:50,139 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:54:50,139 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 16:54:50,139 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:54:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:50,141 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:50,142 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 16:54:50,142 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:50,142 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 16:54:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:50,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:54:50,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:54:50,144 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 16:54:50,144 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:50,144 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:50,144 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,144 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:50,145 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:50,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:50,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:50,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:50,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:50,146 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:50,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,147 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:50,148 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,148 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:50,149 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:50,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:50,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:50,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:50,151 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 16:54:50,152 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:50,151 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:50,152 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:50,152 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:50,152 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:54:50,151 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:50,152 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:50,152 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:50,152 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:50,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:50,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:50,152 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:54:50,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:50,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:50,252 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:50,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:50,256 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,256 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:50,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:50,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:50,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:50,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7ad67c2797888267addece4f08ec7def, UNASSIGN}] 2023-02-13 16:54:50,260 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7ad67c2797888267addece4f08ec7def, UNASSIGN 2023-02-13 16:54:50,261 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=7ad67c2797888267addece4f08ec7def, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:50,261 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676307290261"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307290261"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307290261"}]},"ts":"1676307290261"} 2023-02-13 16:54:50,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 7ad67c2797888267addece4f08ec7def, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:50,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7ad67c2797888267addece4f08ec7def, disabling compactions & flushes 2023-02-13 16:54:50,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:50,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:50,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. after waiting 0 ms 2023-02-13 16:54:50,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:50,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 7ad67c2797888267addece4f08ec7def 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-13 16:54:50,425 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:50,428 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:50,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:50,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:50,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf1/2ee2dfd04bce4b4999d859ab9d2a4dc6 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:50,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:50,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2ee2dfd04bce4b4999d859ab9d2a4dc6, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 16:54:50,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.tmp/cf2/08a50c5a16ab466bb6dc349422858986 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:50,473 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:50,473 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/08a50c5a16ab466bb6dc349422858986, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 16:54:50,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 59ms, sequenceid=103, compaction requested=true 2023-02-13 16:54:50,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-13 16:54:50,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:50,479 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. 2023-02-13 16:54:50,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7ad67c2797888267addece4f08ec7def: Waiting for close lock at 1676307290414Running coprocessor pre-close hooks at 1676307290414Disabling compacts and flushes for region at 1676307290414Disabling writes for close at 1676307290415 (+1 ms)Obtaining lock to block concurrent updates at 1676307290415Preparing flush snapshotting stores in 7ad67c2797888267addece4f08ec7def at 1676307290415Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676307290415Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def. at 1676307290415Flushing 7ad67c2797888267addece4f08ec7def/cf1: creating writer at 1676307290416 (+1 ms)Flushing 7ad67c2797888267addece4f08ec7def/cf1: appending metadata at 1676307290418 (+2 ms)Flushing 7ad67c2797888267addece4f08ec7def/cf1: closing flushed file at 1676307290418Flushing 7ad67c2797888267addece4f08ec7def/cf2: creating writer at 1676307290428 (+10 ms)Flushing 7ad67c2797888267addece4f08ec7def/cf2: appending metadata at 1676307290430 (+2 ms)Flushing 7ad67c2797888267addece4f08ec7def/cf2: closing flushed file at 1676307290430Flushing 7ad67c2797888267addece4f08ec7def/cf1: reopening flushed file at 1676307290442 (+12 ms)Flushing 7ad67c2797888267addece4f08ec7def/cf2: reopening flushed file at 1676307290446 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 7ad67c2797888267addece4f08ec7def in 59ms, sequenceid=103, compaction requested=true at 1676307290474 (+28 ms)Writing region close event to WAL at 1676307290476 (+2 ms)Running coprocessor post-close hooks at 1676307290479 (+3 ms)Closed at 1676307290479 2023-02-13 16:54:50,481 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,481 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=7ad67c2797888267addece4f08ec7def, regionState=CLOSED 2023-02-13 16:54:50,481 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676307290481"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307290481"}]},"ts":"1676307290481"} 2023-02-13 16:54:50,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-13 16:54:50,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 7ad67c2797888267addece4f08ec7def, server=jenkins-hbase4.apache.org,34717,1676307263076 in 220 msec 2023-02-13 16:54:50,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 16:54:50,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7ad67c2797888267addece4f08ec7def, UNASSIGN in 223 msec 2023-02-13 16:54:50,496 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=7ad67c2797888267addece4f08ec7def, threads=16 2023-02-13 16:54:50,496 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/08a50c5a16ab466bb6dc349422858986 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,496 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0b3ef2b5950546d993bd5f1c8e5fb350 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,496 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0cfceef9bb1643c79da10fdb3eea35c0 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,496 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/363c25063fa843e0b9fc36c7c2b909a2 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,496 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/38b99d9751db46d38a4b76d3715644af for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,497 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,497 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,497 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,497 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,497 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/a77ffdb8e67f4c618f5c97b41dfcc713 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,497 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2d6566efde434b7ababfd8a51409f018 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,502 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2ee2dfd04bce4b4999d859ab9d2a4dc6 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,502 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/35dde213e7da43e3ab980e17ecab1352 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,502 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,502 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/940edeefad29403aafea01368c1978ba for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,503 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/b5fd8b19145d4baf8c1d11efee74effc for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,508 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:50,508 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:50,509 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/363c25063fa843e0b9fc36c7c2b909a2, top=false 2023-02-13 16:54:50,509 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:50,509 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/a77ffdb8e67f4c618f5c97b41dfcc713, top=false 2023-02-13 16:54:50,510 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0b3ef2b5950546d993bd5f1c8e5fb350, top=false 2023-02-13 16:54:50,511 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:50,512 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:50,512 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/38b99d9751db46d38a4b76d3715644af, top=true 2023-02-13 16:54:50,516 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/35dde213e7da43e3ab980e17ecab1352, top=false 2023-02-13 16:54:50,516 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b05c187b729402ab0b3cf953e6b76d0 2023-02-13 16:54:50,517 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:50,517 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/08a50c5a16ab466bb6dc349422858986, top=true 2023-02-13 16:54:50,523 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:50,523 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/940edeefad29403aafea01368c1978ba, top=false 2023-02-13 16:54:50,523 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:50,525 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:50,525 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2d6566efde434b7ababfd8a51409f018, top=true 2023-02-13 16:54:50,528 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:50,528 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2ee2dfd04bce4b4999d859ab9d2a4dc6, top=true 2023-02-13 16:54:50,528 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:50,528 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/b5fd8b19145d4baf8c1d11efee74effc, top=true 2023-02-13 16:54:50,534 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-363c25063fa843e0b9fc36c7c2b909a2 for child: 9a11199978f3b74def14fa827748dbd2, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,535 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:50,535 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0cfceef9bb1643c79da10fdb3eea35c0, top=true 2023-02-13 16:54:50,548 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/363c25063fa843e0b9fc36c7c2b909a2 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,548 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,551 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for caafec81ba6249bdb7e6faecf3bea1ce 2023-02-13 16:54:50,565 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0b3ef2b5950546d993bd5f1c8e5fb350 for child: 9a11199978f3b74def14fa827748dbd2, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,565 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-35dde213e7da43e3ab980e17ecab1352 for child: 9a11199978f3b74def14fa827748dbd2, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,565 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-940edeefad29403aafea01368c1978ba for child: 9a11199978f3b74def14fa827748dbd2, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,565 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-a77ffdb8e67f4c618f5c97b41dfcc713 for child: 9a11199978f3b74def14fa827748dbd2, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,570 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0b3ef2b5950546d993bd5f1c8e5fb350 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,570 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,571 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/a77ffdb8e67f4c618f5c97b41dfcc713 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,571 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,576 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-08a50c5a16ab466bb6dc349422858986 for child: 8a9ff403ae971d224cfffc5e5e7ac123, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,576 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/08a50c5a16ab466bb6dc349422858986 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,576 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/f9c336170025405e9269e32fc2a49fdc for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,582 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/f9c336170025405e9269e32fc2a49fdc, top=false 2023-02-13 16:54:50,592 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2ee2dfd04bce4b4999d859ab9d2a4dc6 for child: 8a9ff403ae971d224cfffc5e5e7ac123, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,592 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2ee2dfd04bce4b4999d859ab9d2a4dc6 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,596 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-38b99d9751db46d38a4b76d3715644af for child: 8a9ff403ae971d224cfffc5e5e7ac123, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,596 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/38b99d9751db46d38a4b76d3715644af for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,597 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/940edeefad29403aafea01368c1978ba for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,597 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2d6566efde434b7ababfd8a51409f018 for child: 8a9ff403ae971d224cfffc5e5e7ac123, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,597 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2d6566efde434b7ababfd8a51409f018 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,600 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-b5fd8b19145d4baf8c1d11efee74effc for child: 8a9ff403ae971d224cfffc5e5e7ac123, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,600 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/b5fd8b19145d4baf8c1d11efee74effc for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,600 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0cfceef9bb1643c79da10fdb3eea35c0 for child: 8a9ff403ae971d224cfffc5e5e7ac123, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,600 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0cfceef9bb1643c79da10fdb3eea35c0 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,602 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/35dde213e7da43e3ab980e17ecab1352 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,623 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b05c187b729402ab0b3cf953e6b76d0 2023-02-13 16:54:50,640 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/.splits/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-f9c336170025405e9269e32fc2a49fdc for child: 9a11199978f3b74def14fa827748dbd2, parent: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,646 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/f9c336170025405e9269e32fc2a49fdc for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,660 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for caafec81ba6249bdb7e6faecf3bea1ce 2023-02-13 16:54:50,685 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,685 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,700 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,710 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:50,713 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:51,005 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:51,066 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80 for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:51,092 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:51,093 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e for region: 7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:51,093 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 7ad67c2797888267addece4f08ec7def Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 16:54:51,152 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 16:54:51,154 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 16:54:51,155 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676307291155"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676307291155"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676307291155"}]},"ts":"1676307291155"} 2023-02-13 16:54:51,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307291155"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307291155"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307291155"}]},"ts":"1676307291155"} 2023-02-13 16:54:51,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307291155"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307291155"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307291155"}]},"ts":"1676307291155"} 2023-02-13 16:54:51,159 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a11199978f3b74def14fa827748dbd2, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8a9ff403ae971d224cfffc5e5e7ac123, ASSIGN}] 2023-02-13 16:54:51,160 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8a9ff403ae971d224cfffc5e5e7ac123, ASSIGN 2023-02-13 16:54:51,160 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8a9ff403ae971d224cfffc5e5e7ac123, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:51,161 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a11199978f3b74def14fa827748dbd2, ASSIGN 2023-02-13 16:54:51,162 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a11199978f3b74def14fa827748dbd2, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:51,311 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 16:54:51,312 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=8a9ff403ae971d224cfffc5e5e7ac123, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:51,313 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307291312"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307291312"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307291312"}]},"ts":"1676307291312"} 2023-02-13 16:54:51,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure 8a9ff403ae971d224cfffc5e5e7ac123, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:51,316 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=9a11199978f3b74def14fa827748dbd2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:51,316 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307291316"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307291316"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307291316"}]},"ts":"1676307291316"} 2023-02-13 16:54:51,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure 9a11199978f3b74def14fa827748dbd2, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:51,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9a11199978f3b74def14fa827748dbd2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.', STARTKEY => '', ENDKEY => '60'} 2023-02-13 16:54:51,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:51,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:51,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:51,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:51,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:51,470 INFO [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:51,471 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1 2023-02-13 16:54:51,471 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1 2023-02-13 16:54:51,471 INFO [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a11199978f3b74def14fa827748dbd2 columnFamilyName cf1 2023-02-13 16:54:51,481 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47-bottom 2023-02-13 16:54:51,484 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:51,485 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce-bottom 2023-02-13 16:54:51,489 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80-bottom 2023-02-13 16:54:51,493 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35-bottom 2023-02-13 16:54:51,495 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-35dde213e7da43e3ab980e17ecab1352 2023-02-13 16:54:51,498 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-940edeefad29403aafea01368c1978ba 2023-02-13 16:54:51,501 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-f9c336170025405e9269e32fc2a49fdc 2023-02-13 16:54:51,501 INFO [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(310): Store=9a11199978f3b74def14fa827748dbd2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:51,501 INFO [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:51,502 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2 2023-02-13 16:54:51,502 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2 2023-02-13 16:54:51,502 INFO [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a11199978f3b74def14fa827748dbd2 columnFamilyName cf2 2023-02-13 16:54:51,512 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34-bottom 2023-02-13 16:54:51,516 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5-bottom 2023-02-13 16:54:51,520 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e-bottom 2023-02-13 16:54:51,523 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:51,524 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0-bottom 2023-02-13 16:54:51,527 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0b3ef2b5950546d993bd5f1c8e5fb350 2023-02-13 16:54:51,530 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-363c25063fa843e0b9fc36c7c2b909a2 2023-02-13 16:54:51,532 DEBUG [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-a77ffdb8e67f4c618f5c97b41dfcc713 2023-02-13 16:54:51,533 INFO [StoreOpener-9a11199978f3b74def14fa827748dbd2-1] regionserver.HStore(310): Store=9a11199978f3b74def14fa827748dbd2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:51,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:51,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:51,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:51,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:51,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9a11199978f3b74def14fa827748dbd2; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73809976, jitterRate=0.09985435009002686}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:51,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9a11199978f3b74def14fa827748dbd2: Running coprocessor pre-open hook at 1676307291469Writing region info on filesystem at 1676307291469Initializing all the Stores at 1676307291470 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307291470Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307291470Cleaning up temporary data from old regions at 1676307291535 (+65 ms)Cleaning up detritus from prior splits at 1676307291535Running coprocessor post-open hooks at 1676307291538 (+3 ms)Region opened successfully at 1676307291538 2023-02-13 16:54:51,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2., pid=78, masterSystemTime=1676307291465 2023-02-13 16:54:51,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:51,541 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:51,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:54:51,541 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:51,541 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,541 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1912): 9a11199978f3b74def14fa827748dbd2/cf1 is initiating minor compaction (all files) 2023-02-13 16:54:51,541 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,541 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1912): 9a11199978f3b74def14fa827748dbd2/cf2 is initiating minor compaction (all files) 2023-02-13 16:54:51,541 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9a11199978f3b74def14fa827748dbd2/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,541 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 9a11199978f3b74def14fa827748dbd2/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,541 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-940edeefad29403aafea01368c1978ba, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-35dde213e7da43e3ab980e17ecab1352, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-f9c336170025405e9269e32fc2a49fdc, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/.tmp, totalSize=35.0 K 2023-02-13 16:54:51,541 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0b3ef2b5950546d993bd5f1c8e5fb350, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-a77ffdb8e67f4c618f5c97b41dfcc713, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-363c25063fa843e0b9fc36c7c2b909a2, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/.tmp, totalSize=35.0 K 2023-02-13 16:54:51,542 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307288596 2023-02-13 16:54:51,542 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307288596 2023-02-13 16:54:51,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,542 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307288786 2023-02-13 16:54:51,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8a9ff403ae971d224cfffc5e5e7ac123, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.', STARTKEY => '60', ENDKEY => ''} 2023-02-13 16:54:51,543 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=9a11199978f3b74def14fa827748dbd2, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:51,543 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307291543"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307291543"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307291543"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307291543"}]},"ts":"1676307291543"} 2023-02-13 16:54:51,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:51,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:51,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:51,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:51,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:51,544 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307288786 2023-02-13 16:54:51,545 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307288976 2023-02-13 16:54:51,546 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307288976 2023-02-13 16:54:51,546 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-940edeefad29403aafea01368c1978ba, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307289170 2023-02-13 16:54:51,546 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0b3ef2b5950546d993bd5f1c8e5fb350, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307289170 2023-02-13 16:54:51,547 INFO [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:51,547 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-35dde213e7da43e3ab980e17ecab1352, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307289383 2023-02-13 16:54:51,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-02-13 16:54:51,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure 9a11199978f3b74def14fa827748dbd2, server=jenkins-hbase4.apache.org,34717,1676307263076 in 228 msec 2023-02-13 16:54:51,547 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-f9c336170025405e9269e32fc2a49fdc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307289563 2023-02-13 16:54:51,548 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-a77ffdb8e67f4c618f5c97b41dfcc713, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307289383 2023-02-13 16:54:51,548 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307289741 2023-02-13 16:54:51,548 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1 2023-02-13 16:54:51,548 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-363c25063fa843e0b9fc36c7c2b909a2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307289563 2023-02-13 16:54:51,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a11199978f3b74def14fa827748dbd2, ASSIGN in 388 msec 2023-02-13 16:54:51,548 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1 2023-02-13 16:54:51,549 INFO [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a9ff403ae971d224cfffc5e5e7ac123 columnFamilyName cf1 2023-02-13 16:54:51,549 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307289741 2023-02-13 16:54:51,561 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47-top 2023-02-13 16:54:51,563 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] throttle.PressureAwareThroughputController(145): 9a11199978f3b74def14fa827748dbd2#cf2#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:51,565 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:51,565 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce-top 2023-02-13 16:54:51,569 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9a11199978f3b74def14fa827748dbd2#cf1#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:51,575 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80-top 2023-02-13 16:54:51,578 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35-top 2023-02-13 16:54:51,585 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:51,585 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:51,589 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:51,589 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:51,591 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/.tmp/cf2/9b313c30af0a47e584295288c4c7a510 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/9b313c30af0a47e584295288c4c7a510 2023-02-13 16:54:51,592 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/.tmp/cf1/de2bb094a51c495e9eef041695cd32c6 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/de2bb094a51c495e9eef041695cd32c6 2023-02-13 16:54:51,593 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:51,593 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:51,593 INFO [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(310): Store=8a9ff403ae971d224cfffc5e5e7ac123/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:51,593 INFO [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:51,594 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2 2023-02-13 16:54:51,594 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2 2023-02-13 16:54:51,594 INFO [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a9ff403ae971d224cfffc5e5e7ac123 columnFamilyName cf2 2023-02-13 16:54:51,598 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9a11199978f3b74def14fa827748dbd2/cf2 of 9a11199978f3b74def14fa827748dbd2 into 9b313c30af0a47e584295288c4c7a510(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:51,598 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9a11199978f3b74def14fa827748dbd2/cf1 of 9a11199978f3b74def14fa827748dbd2 into de2bb094a51c495e9eef041695cd32c6(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:51,598 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9a11199978f3b74def14fa827748dbd2: 2023-02-13 16:54:51,598 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9a11199978f3b74def14fa827748dbd2: 2023-02-13 16:54:51,598 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2., storeName=9a11199978f3b74def14fa827748dbd2/cf2, priority=9, startTime=1676307291541; duration=0sec 2023-02-13 16:54:51,598 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2., storeName=9a11199978f3b74def14fa827748dbd2/cf1, priority=9, startTime=1676307291539; duration=0sec 2023-02-13 16:54:51,598 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:51,598 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:51,605 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34-top 2023-02-13 16:54:51,608 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5-top 2023-02-13 16:54:51,611 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e-top 2023-02-13 16:54:51,614 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:51,615 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0-top 2023-02-13 16:54:51,618 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:51,618 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:51,621 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:51,621 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:51,625 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:51,625 DEBUG [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:51,625 INFO [StoreOpener-8a9ff403ae971d224cfffc5e5e7ac123-1] regionserver.HStore(310): Store=8a9ff403ae971d224cfffc5e5e7ac123/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:51,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:51,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:51,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:51,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:51,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8a9ff403ae971d224cfffc5e5e7ac123; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59879807, jitterRate=-0.10772134363651276}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:51,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8a9ff403ae971d224cfffc5e5e7ac123: Running coprocessor pre-open hook at 1676307291544Writing region info on filesystem at 1676307291544Initializing all the Stores at 1676307291546 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307291546Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307291546Cleaning up temporary data from old regions at 1676307291627 (+81 ms)Cleaning up detritus from prior splits at 1676307291628 (+1 ms)Running coprocessor post-open hooks at 1676307291630 (+2 ms)Region opened successfully at 1676307291630 2023-02-13 16:54:51,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123., pid=77, masterSystemTime=1676307291465 2023-02-13 16:54:51,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:51,632 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:51,632 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:54:51,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:51,632 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,632 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,632 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1912): 8a9ff403ae971d224cfffc5e5e7ac123/cf2 is initiating minor compaction (all files) 2023-02-13 16:54:51,632 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1912): 8a9ff403ae971d224cfffc5e5e7ac123/cf1 is initiating minor compaction (all files) 2023-02-13 16:54:51,632 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8a9ff403ae971d224cfffc5e5e7ac123/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,633 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8a9ff403ae971d224cfffc5e5e7ac123/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,633 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-38b99d9751db46d38a4b76d3715644af, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0cfceef9bb1643c79da10fdb3eea35c0, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-08a50c5a16ab466bb6dc349422858986] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/.tmp, totalSize=34.8 K 2023-02-13 16:54:51,633 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2d6566efde434b7ababfd8a51409f018, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-b5fd8b19145d4baf8c1d11efee74effc, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2ee2dfd04bce4b4999d859ab9d2a4dc6] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/.tmp, totalSize=34.8 K 2023-02-13 16:54:51,633 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307288596 2023-02-13 16:54:51,634 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting 928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307288596 2023-02-13 16:54:51,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,634 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307288786 2023-02-13 16:54:51,634 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307288786 2023-02-13 16:54:51,634 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=8a9ff403ae971d224cfffc5e5e7ac123, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:51,634 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307288976 2023-02-13 16:54:51,635 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307288976 2023-02-13 16:54:51,635 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307291634"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307291634"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307291634"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307291634"}]},"ts":"1676307291634"} 2023-02-13 16:54:51,635 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676307289741 2023-02-13 16:54:51,635 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676307289741 2023-02-13 16:54:51,636 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-38b99d9751db46d38a4b76d3715644af, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 16:54:51,636 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2d6566efde434b7ababfd8a51409f018, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 16:54:51,636 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0cfceef9bb1643c79da10fdb3eea35c0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 16:54:51,636 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-b5fd8b19145d4baf8c1d11efee74effc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 16:54:51,636 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-08a50c5a16ab466bb6dc349422858986, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 16:54:51,637 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2ee2dfd04bce4b4999d859ab9d2a4dc6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 16:54:51,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-13 16:54:51,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure 8a9ff403ae971d224cfffc5e5e7ac123, server=jenkins-hbase4.apache.org,34717,1676307263076 in 322 msec 2023-02-13 16:54:51,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-13 16:54:51,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8a9ff403ae971d224cfffc5e5e7ac123, ASSIGN in 480 msec 2023-02-13 16:54:51,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 in 1.3850 sec 2023-02-13 16:54:51,654 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] throttle.PressureAwareThroughputController(145): 8a9ff403ae971d224cfffc5e5e7ac123#cf2#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:51,661 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8a9ff403ae971d224cfffc5e5e7ac123#cf1#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:51,676 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 244fadfdcb6241579dbb267da523a7d9: NONE, but ROW specified in column family configuration 2023-02-13 16:54:51,677 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/.tmp/cf2/244fadfdcb6241579dbb267da523a7d9 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/244fadfdcb6241579dbb267da523a7d9 2023-02-13 16:54:51,679 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for db3b9168716e4d739fee27155b226fdb: NONE, but ROW specified in column family configuration 2023-02-13 16:54:51,679 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/.tmp/cf1/db3b9168716e4d739fee27155b226fdb as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/db3b9168716e4d739fee27155b226fdb 2023-02-13 16:54:51,682 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 244fadfdcb6241579dbb267da523a7d9: NONE, but ROW specified in column family configuration 2023-02-13 16:54:51,683 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8a9ff403ae971d224cfffc5e5e7ac123/cf2 of 8a9ff403ae971d224cfffc5e5e7ac123 into 244fadfdcb6241579dbb267da523a7d9(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:51,683 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8a9ff403ae971d224cfffc5e5e7ac123: 2023-02-13 16:54:51,683 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123., storeName=8a9ff403ae971d224cfffc5e5e7ac123/cf2, priority=9, startTime=1676307291632; duration=0sec 2023-02-13 16:54:51,683 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:51,684 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for db3b9168716e4d739fee27155b226fdb: NONE, but ROW specified in column family configuration 2023-02-13 16:54:51,685 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8a9ff403ae971d224cfffc5e5e7ac123/cf1 of 8a9ff403ae971d224cfffc5e5e7ac123 into db3b9168716e4d739fee27155b226fdb(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:51,685 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8a9ff403ae971d224cfffc5e5e7ac123: 2023-02-13 16:54:51,685 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123., storeName=8a9ff403ae971d224cfffc5e5e7ac123/cf1, priority=9, startTime=1676307291631; duration=0sec 2023-02-13 16:54:51,685 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:51,767 DEBUG [Listener at localhost/38547] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 16:54:51,772 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47024, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 16:54:51,772 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,773 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 16:54:51,774 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HStore(1912): 9a11199978f3b74def14fa827748dbd2/cf1 is initiating major compaction (all files) 2023-02-13 16:54:51,774 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@233ac1f0; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:51,774 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9a11199978f3b74def14fa827748dbd2/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,774 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 16:54:51,774 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HStore(1912): 9a11199978f3b74def14fa827748dbd2/cf2 is initiating major compaction (all files) 2023-02-13 16:54:51,774 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/de2bb094a51c495e9eef041695cd32c6] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/.tmp, totalSize=6.8 K 2023-02-13 16:54:51,774 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@78f5fce; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:51,774 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 9a11199978f3b74def14fa827748dbd2/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:51,774 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/9b313c30af0a47e584295288c4c7a510] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/.tmp, totalSize=6.8 K 2023-02-13 16:54:51,774 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting de2bb094a51c495e9eef041695cd32c6, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307288596 2023-02-13 16:54:51,774 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 9b313c30af0a47e584295288c4c7a510, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307288596 2023-02-13 16:54:51,776 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,776 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 16:54:51,776 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.HStore(1912): 8a9ff403ae971d224cfffc5e5e7ac123/cf1 is initiating major compaction (all files) 2023-02-13 16:54:51,776 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4dcbcd60; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:54:51,776 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 16:54:51,776 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.HStore(1912): 8a9ff403ae971d224cfffc5e5e7ac123/cf2 is initiating major compaction (all files) 2023-02-13 16:54:51,776 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@29e6702a; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 16:54:51,777 INFO [Listener at localhost/38547] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 16:54:51,779 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] throttle.PressureAwareThroughputController(145): 9a11199978f3b74def14fa827748dbd2#cf2#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:51,780 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9a11199978f3b74def14fa827748dbd2#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:51,789 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. is not mergeable because it has references 2023-02-13 16:54:51,789 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. is not mergeable because it has references 2023-02-13 16:54:51,793 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/.tmp/cf1/ca1ec89180e94be99d23791fb43ec5aa as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/ca1ec89180e94be99d23791fb43ec5aa 2023-02-13 16:54:51,796 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/.tmp/cf2/75240ef825804706a93f70c93cf98a7a as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/75240ef825804706a93f70c93cf98a7a 2023-02-13 16:54:51,798 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 9a11199978f3b74def14fa827748dbd2/cf1 of 9a11199978f3b74def14fa827748dbd2 into ca1ec89180e94be99d23791fb43ec5aa(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:51,798 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9a11199978f3b74def14fa827748dbd2: 2023-02-13 16:54:51,799 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676307291774; duration=0sec 2023-02-13 16:54:51,799 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 16:54:51,799 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8a9ff403ae971d224cfffc5e5e7ac123/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,799 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/db3b9168716e4d739fee27155b226fdb] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/.tmp, totalSize=5.0 K 2023-02-13 16:54:51,800 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting db3b9168716e4d739fee27155b226fdb, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 16:54:51,800 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 9a11199978f3b74def14fa827748dbd2/cf2 of 9a11199978f3b74def14fa827748dbd2 into 75240ef825804706a93f70c93cf98a7a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:51,800 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9a11199978f3b74def14fa827748dbd2: 2023-02-13 16:54:51,800 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676307291774; duration=0sec 2023-02-13 16:54:51,800 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:54:51,800 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8a9ff403ae971d224cfffc5e5e7ac123/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:51,800 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/244fadfdcb6241579dbb267da523a7d9] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/.tmp, totalSize=5.0 K 2023-02-13 16:54:51,801 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting 244fadfdcb6241579dbb267da523a7d9, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 16:54:51,802 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8a9ff403ae971d224cfffc5e5e7ac123#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:51,803 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] throttle.PressureAwareThroughputController(145): 8a9ff403ae971d224cfffc5e5e7ac123#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:51,897 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. is not mergeable because it has references 2023-02-13 16:54:51,897 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. is not mergeable because it has references 2023-02-13 16:54:52,005 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. is not mergeable because it has references 2023-02-13 16:54:52,006 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. is not mergeable because it has references 2023-02-13 16:54:52,114 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. is not mergeable because it has references 2023-02-13 16:54:52,114 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. is not mergeable because it has references 2023-02-13 16:54:52,222 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3ff6e3d9c9494f61975150f6800bb69f: NONE, but ROW specified in column family configuration 2023-02-13 16:54:52,222 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4fa0b5b5f3144aa1a6cfb1c017727c2c: NONE, but ROW specified in column family configuration 2023-02-13 16:54:52,223 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/.tmp/cf1/3ff6e3d9c9494f61975150f6800bb69f as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/3ff6e3d9c9494f61975150f6800bb69f 2023-02-13 16:54:52,223 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. is not mergeable because it has references 2023-02-13 16:54:52,223 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/.tmp/cf2/4fa0b5b5f3144aa1a6cfb1c017727c2c as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/4fa0b5b5f3144aa1a6cfb1c017727c2c 2023-02-13 16:54:52,224 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. is not mergeable because it has references 2023-02-13 16:54:52,227 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3ff6e3d9c9494f61975150f6800bb69f: NONE, but ROW specified in column family configuration 2023-02-13 16:54:52,228 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4fa0b5b5f3144aa1a6cfb1c017727c2c: NONE, but ROW specified in column family configuration 2023-02-13 16:54:52,228 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 8a9ff403ae971d224cfffc5e5e7ac123/cf1 of 8a9ff403ae971d224cfffc5e5e7ac123 into 3ff6e3d9c9494f61975150f6800bb69f(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:52,228 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8a9ff403ae971d224cfffc5e5e7ac123: 2023-02-13 16:54:52,228 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676307291776; duration=0sec 2023-02-13 16:54:52,228 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:52,229 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 8a9ff403ae971d224cfffc5e5e7ac123/cf2 of 8a9ff403ae971d224cfffc5e5e7ac123 into 4fa0b5b5f3144aa1a6cfb1c017727c2c(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:52,229 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8a9ff403ae971d224cfffc5e5e7ac123: 2023-02-13 16:54:52,229 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676307291776; duration=0sec 2023-02-13 16:54:52,229 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:52,332 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. is not mergeable because it has references 2023-02-13 16:54:52,332 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34717] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. is not mergeable because it has references 2023-02-13 16:54:52,338 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-13 16:54:52,339 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:52,341 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,341 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 16:54:52,345 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307292344"}]},"ts":"1676307292344"} 2023-02-13 16:54:52,347 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-13 16:54:52,349 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-13 16:54:52,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a11199978f3b74def14fa827748dbd2, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8a9ff403ae971d224cfffc5e5e7ac123, UNASSIGN}] 2023-02-13 16:54:52,350 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8a9ff403ae971d224cfffc5e5e7ac123, UNASSIGN 2023-02-13 16:54:52,351 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=8a9ff403ae971d224cfffc5e5e7ac123, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:52,351 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307292351"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307292351"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307292351"}]},"ts":"1676307292351"} 2023-02-13 16:54:52,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 8a9ff403ae971d224cfffc5e5e7ac123, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:52,353 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a11199978f3b74def14fa827748dbd2, UNASSIGN 2023-02-13 16:54:52,354 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=9a11199978f3b74def14fa827748dbd2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:52,354 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307292354"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307292354"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307292354"}]},"ts":"1676307292354"} 2023-02-13 16:54:52,355 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 9a11199978f3b74def14fa827748dbd2, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:52,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 16:54:52,503 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:52,503 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8a9ff403ae971d224cfffc5e5e7ac123, disabling compactions & flushes 2023-02-13 16:54:52,503 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:52,503 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:52,503 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. after waiting 0 ms 2023-02-13 16:54:52,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:52,504 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2d6566efde434b7ababfd8a51409f018, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-b5fd8b19145d4baf8c1d11efee74effc, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/db3b9168716e4d739fee27155b226fdb, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2ee2dfd04bce4b4999d859ab9d2a4dc6] to archive 2023-02-13 16:54:52,505 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:52,507 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,508 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,509 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,510 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,511 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2d6566efde434b7ababfd8a51409f018 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:52,511 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-b5fd8b19145d4baf8c1d11efee74effc to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:52,512 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/db3b9168716e4d739fee27155b226fdb to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/db3b9168716e4d739fee27155b226fdb 2023-02-13 16:54:52,513 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2ee2dfd04bce4b4999d859ab9d2a4dc6 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:52,514 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-38b99d9751db46d38a4b76d3715644af, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0cfceef9bb1643c79da10fdb3eea35c0, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/244fadfdcb6241579dbb267da523a7d9, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-08a50c5a16ab466bb6dc349422858986] to archive 2023-02-13 16:54:52,515 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:52,516 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,517 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,518 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,519 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,520 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-38b99d9751db46d38a4b76d3715644af to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:52,521 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0cfceef9bb1643c79da10fdb3eea35c0 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:52,522 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/244fadfdcb6241579dbb267da523a7d9 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/244fadfdcb6241579dbb267da523a7d9 2023-02-13 16:54:52,523 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-08a50c5a16ab466bb6dc349422858986 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:52,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 16:54:52,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:52,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123. 2023-02-13 16:54:52,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8a9ff403ae971d224cfffc5e5e7ac123: Waiting for close lock at 1676307292503Running coprocessor pre-close hooks at 1676307292503Disabling compacts and flushes for region at 1676307292503Disabling writes for close at 1676307292504 (+1 ms)Writing region close event to WAL at 1676307292524 (+20 ms)Running coprocessor post-close hooks at 1676307292526 (+2 ms)Closed at 1676307292526 2023-02-13 16:54:52,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:52,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:52,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9a11199978f3b74def14fa827748dbd2, disabling compactions & flushes 2023-02-13 16:54:52,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:52,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:52,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. after waiting 0 ms 2023-02-13 16:54:52,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:52,528 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=8a9ff403ae971d224cfffc5e5e7ac123, regionState=CLOSED 2023-02-13 16:54:52,528 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307292528"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307292528"}]},"ts":"1676307292528"} 2023-02-13 16:54:52,529 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-940edeefad29403aafea01368c1978ba, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-35dde213e7da43e3ab980e17ecab1352, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-f9c336170025405e9269e32fc2a49fdc, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/de2bb094a51c495e9eef041695cd32c6, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce-bottom] to archive 2023-02-13 16:54:52,530 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:52,531 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/928a7b3e745d4bf8b0152f6117c6bc47.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-13 16:54:52,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 8a9ff403ae971d224cfffc5e5e7ac123, server=jenkins-hbase4.apache.org,34717,1676307263076 in 178 msec 2023-02-13 16:54:52,532 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/edc584f92e74472899a980e41ef24c35.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8a9ff403ae971d224cfffc5e5e7ac123, UNASSIGN in 182 msec 2023-02-13 16:54:52,533 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/e0bad3d69ee84f9a81d996df11eb6f80.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,534 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-940edeefad29403aafea01368c1978ba to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-940edeefad29403aafea01368c1978ba 2023-02-13 16:54:52,535 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-35dde213e7da43e3ab980e17ecab1352 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-35dde213e7da43e3ab980e17ecab1352 2023-02-13 16:54:52,537 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-f9c336170025405e9269e32fc2a49fdc to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-f9c336170025405e9269e32fc2a49fdc 2023-02-13 16:54:52,538 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/de2bb094a51c495e9eef041695cd32c6 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/de2bb094a51c495e9eef041695cd32c6 2023-02-13 16:54:52,539 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/caafec81ba6249bdb7e6faecf3bea1ce.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,540 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0b3ef2b5950546d993bd5f1c8e5fb350, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-a77ffdb8e67f4c618f5c97b41dfcc713, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-363c25063fa843e0b9fc36c7c2b909a2, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/9b313c30af0a47e584295288c4c7a510, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0-bottom] to archive 2023-02-13 16:54:52,541 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:52,542 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/600ac40c07764deeaee534f86d1434d5.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,543 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/77139cb766334e39891ea13c6b6b987e.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,544 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/4941ac96b48147d0bc56e9194304db34.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,545 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0b3ef2b5950546d993bd5f1c8e5fb350 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-0b3ef2b5950546d993bd5f1c8e5fb350 2023-02-13 16:54:52,546 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-a77ffdb8e67f4c618f5c97b41dfcc713 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-a77ffdb8e67f4c618f5c97b41dfcc713 2023-02-13 16:54:52,547 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-363c25063fa843e0b9fc36c7c2b909a2 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/testSplitTableRegionDeletedRowsDaughter=7ad67c2797888267addece4f08ec7def-363c25063fa843e0b9fc36c7c2b909a2 2023-02-13 16:54:52,547 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/9b313c30af0a47e584295288c4c7a510 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/9b313c30af0a47e584295288c4c7a510 2023-02-13 16:54:52,548 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/7b05c187b729402ab0b3cf953e6b76d0.7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 16:54:52,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:52,552 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2. 2023-02-13 16:54:52,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9a11199978f3b74def14fa827748dbd2: Waiting for close lock at 1676307292528Running coprocessor pre-close hooks at 1676307292528Disabling compacts and flushes for region at 1676307292528Disabling writes for close at 1676307292528Writing region close event to WAL at 1676307292549 (+21 ms)Running coprocessor post-close hooks at 1676307292552 (+3 ms)Closed at 1676307292552 2023-02-13 16:54:52,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:52,553 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=9a11199978f3b74def14fa827748dbd2, regionState=CLOSED 2023-02-13 16:54:52,554 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676307292553"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307292553"}]},"ts":"1676307292553"} 2023-02-13 16:54:52,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-13 16:54:52,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 9a11199978f3b74def14fa827748dbd2, server=jenkins-hbase4.apache.org,34717,1676307263076 in 200 msec 2023-02-13 16:54:52,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-13 16:54:52,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a11199978f3b74def14fa827748dbd2, UNASSIGN in 206 msec 2023-02-13 16:54:52,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307292558"}]},"ts":"1676307292558"} 2023-02-13 16:54:52,559 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-13 16:54:52,564 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-13 16:54:52,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 224 msec 2023-02-13 16:54:52,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 16:54:52,650 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-13 16:54:52,651 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,653 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,654 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 16:54:52,658 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,658 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:52,658 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:52,660 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/recovered.edits] 2023-02-13 16:54:52,660 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/recovered.edits] 2023-02-13 16:54:52,664 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/recovered.edits] 2023-02-13 16:54:52,666 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2d6566efde434b7ababfd8a51409f018 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2d6566efde434b7ababfd8a51409f018 2023-02-13 16:54:52,667 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/3ff6e3d9c9494f61975150f6800bb69f to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf1/3ff6e3d9c9494f61975150f6800bb69f 2023-02-13 16:54:52,667 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/ca1ec89180e94be99d23791fb43ec5aa to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf1/ca1ec89180e94be99d23791fb43ec5aa 2023-02-13 16:54:52,668 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2ee2dfd04bce4b4999d859ab9d2a4dc6 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/2ee2dfd04bce4b4999d859ab9d2a4dc6 2023-02-13 16:54:52,669 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/35dde213e7da43e3ab980e17ecab1352 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/35dde213e7da43e3ab980e17ecab1352 2023-02-13 16:54:52,669 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/4fa0b5b5f3144aa1a6cfb1c017727c2c to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/cf2/4fa0b5b5f3144aa1a6cfb1c017727c2c 2023-02-13 16:54:52,669 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/75240ef825804706a93f70c93cf98a7a to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/cf2/75240ef825804706a93f70c93cf98a7a 2023-02-13 16:54:52,670 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/928a7b3e745d4bf8b0152f6117c6bc47 2023-02-13 16:54:52,672 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/recovered.edits/113.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123/recovered.edits/113.seqid 2023-02-13 16:54:52,672 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/940edeefad29403aafea01368c1978ba to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/940edeefad29403aafea01368c1978ba 2023-02-13 16:54:52,672 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/recovered.edits/113.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2/recovered.edits/113.seqid 2023-02-13 16:54:52,672 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:54:52,673 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a11199978f3b74def14fa827748dbd2 2023-02-13 16:54:52,673 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/b5fd8b19145d4baf8c1d11efee74effc to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/b5fd8b19145d4baf8c1d11efee74effc 2023-02-13 16:54:52,674 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/caafec81ba6249bdb7e6faecf3bea1ce 2023-02-13 16:54:52,675 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/e0bad3d69ee84f9a81d996df11eb6f80 2023-02-13 16:54:52,676 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/edc584f92e74472899a980e41ef24c35 2023-02-13 16:54:52,677 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/f9c336170025405e9269e32fc2a49fdc to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf1/f9c336170025405e9269e32fc2a49fdc 2023-02-13 16:54:52,680 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/08a50c5a16ab466bb6dc349422858986 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/08a50c5a16ab466bb6dc349422858986 2023-02-13 16:54:52,681 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0b3ef2b5950546d993bd5f1c8e5fb350 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0b3ef2b5950546d993bd5f1c8e5fb350 2023-02-13 16:54:52,682 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0cfceef9bb1643c79da10fdb3eea35c0 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/0cfceef9bb1643c79da10fdb3eea35c0 2023-02-13 16:54:52,683 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/363c25063fa843e0b9fc36c7c2b909a2 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/363c25063fa843e0b9fc36c7c2b909a2 2023-02-13 16:54:52,684 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/38b99d9751db46d38a4b76d3715644af to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/38b99d9751db46d38a4b76d3715644af 2023-02-13 16:54:52,685 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/4941ac96b48147d0bc56e9194304db34 2023-02-13 16:54:52,686 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/600ac40c07764deeaee534f86d1434d5 2023-02-13 16:54:52,687 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/77139cb766334e39891ea13c6b6b987e 2023-02-13 16:54:52,688 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/7b05c187b729402ab0b3cf953e6b76d0 2023-02-13 16:54:52,689 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/a77ffdb8e67f4c618f5c97b41dfcc713 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/cf2/a77ffdb8e67f4c618f5c97b41dfcc713 2023-02-13 16:54:52,691 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/recovered.edits/106.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def/recovered.edits/106.seqid 2023-02-13 16:54:52,692 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7ad67c2797888267addece4f08ec7def 2023-02-13 16:54:52,692 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 16:54:52,694 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,696 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-13 16:54:52,697 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-13 16:54:52,698 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,698 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-13 16:54:52,698 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307292698"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:52,699 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307292698"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:52,699 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307292698"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:52,700 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 16:54:52,700 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7ad67c2797888267addece4f08ec7def, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676307288092.7ad67c2797888267addece4f08ec7def.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 9a11199978f3b74def14fa827748dbd2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676307290256.9a11199978f3b74def14fa827748dbd2.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 8a9ff403ae971d224cfffc5e5e7ac123, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676307290256.8a9ff403ae971d224cfffc5e5e7ac123.', STARTKEY => '60', ENDKEY => ''}] 2023-02-13 16:54:52,700 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-13 16:54:52,700 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307292700"}]},"ts":"9223372036854775807"} 2023-02-13 16:54:52,701 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-13 16:54:52,703 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:54:52,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 52 msec 2023-02-13 16:54:52,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 16:54:52,755 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-13 16:54:52,770 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=405 (was 400) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-18 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53020 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:40556 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:43262 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:43768 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:40576 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53000 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53616 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:40512 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:52982 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:40752 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26bb8ba6-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26bb8ba6-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53844 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:43726 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:53004 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:40258 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:40274 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-189343716_15 at /127.0.0.1:43240 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=854 (was 864), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=210 (was 227), ProcessCount=168 (was 168), AvailableMemoryMB=1258 (was 1327) 2023-02-13 16:54:52,788 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=405, OpenFileDescriptor=854, MaxFileDescriptor=60000, SystemLoadAverage=210, ProcessCount=168, AvailableMemoryMB=1257 2023-02-13 16:54:52,788 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:54:52,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:54:52,792 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:54:52,793 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:54:52,795 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:52,796 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:52,797 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:52,798 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897 empty. 2023-02-13 16:54:52,798 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:52,798 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 16:54:52,811 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:52,812 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 60a6f7bdba888b67b70e7783b1b80897, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:54:52,825 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:52,825 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 60a6f7bdba888b67b70e7783b1b80897, disabling compactions & flushes 2023-02-13 16:54:52,825 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:52,825 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:52,825 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. after waiting 0 ms 2023-02-13 16:54:52,825 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:52,825 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:52,825 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 60a6f7bdba888b67b70e7783b1b80897: Waiting for close lock at 1676307292825Disabling compacts and flushes for region at 1676307292825Disabling writes for close at 1676307292825Writing region close event to WAL at 1676307292825Closed at 1676307292825 2023-02-13 16:54:52,827 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:52,828 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307292828"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307292828"}]},"ts":"1676307292828"} 2023-02-13 16:54:52,829 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:52,830 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:52,830 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307292830"}]},"ts":"1676307292830"} 2023-02-13 16:54:52,831 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-13 16:54:52,834 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:54:52,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:54:52,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:54:52,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:54:52,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:54:52,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, ASSIGN}] 2023-02-13 16:54:52,838 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, ASSIGN 2023-02-13 16:54:52,838 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:54:52,988 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:54:52,991 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=60a6f7bdba888b67b70e7783b1b80897, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:52,991 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307292991"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307292991"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307292991"}]},"ts":"1676307292991"} 2023-02-13 16:54:52,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 60a6f7bdba888b67b70e7783b1b80897, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:53,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 60a6f7bdba888b67b70e7783b1b80897, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:53,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:53,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:53,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,149 INFO [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,150 DEBUG [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/cf1 2023-02-13 16:54:53,150 DEBUG [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/cf1 2023-02-13 16:54:53,151 INFO [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60a6f7bdba888b67b70e7783b1b80897 columnFamilyName cf1 2023-02-13 16:54:53,151 INFO [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] regionserver.HStore(310): Store=60a6f7bdba888b67b70e7783b1b80897/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:53,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:53,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 60a6f7bdba888b67b70e7783b1b80897; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59427061, jitterRate=-0.11446778476238251}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:53,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 60a6f7bdba888b67b70e7783b1b80897: Running coprocessor pre-open hook at 1676307293148Writing region info on filesystem at 1676307293148Initializing all the Stores at 1676307293149 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307293149Cleaning up temporary data from old regions at 1676307293153 (+4 ms)Cleaning up detritus from prior splits at 1676307293153Running coprocessor post-open hooks at 1676307293156 (+3 ms)Region opened successfully at 1676307293156 2023-02-13 16:54:53,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897., pid=87, masterSystemTime=1676307293144 2023-02-13 16:54:53,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,159 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=60a6f7bdba888b67b70e7783b1b80897, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:53,159 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307293159"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307293159"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307293159"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307293159"}]},"ts":"1676307293159"} 2023-02-13 16:54:53,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-13 16:54:53,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 60a6f7bdba888b67b70e7783b1b80897, server=jenkins-hbase4.apache.org,34717,1676307263076 in 167 msec 2023-02-13 16:54:53,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-13 16:54:53,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, ASSIGN in 327 msec 2023-02-13 16:54:53,164 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:53,164 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307293164"}]},"ts":"1676307293164"} 2023-02-13 16:54:53,165 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-13 16:54:53,171 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:53,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 378 msec 2023-02-13 16:54:53,304 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 16:54:53,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:54:53,324 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 16:54:53,325 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:53,325 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 16:54:53,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-13 16:54:53,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, REOPEN/MOVE}] 2023-02-13 16:54:53,342 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, REOPEN/MOVE 2023-02-13 16:54:53,342 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=60a6f7bdba888b67b70e7783b1b80897, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:53,342 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307293342"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307293342"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307293342"}]},"ts":"1676307293342"} 2023-02-13 16:54:53,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 60a6f7bdba888b67b70e7783b1b80897, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:53,496 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 60a6f7bdba888b67b70e7783b1b80897, disabling compactions & flushes 2023-02-13 16:54:53,497 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. after waiting 0 ms 2023-02-13 16:54:53,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,500 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 16:54:53,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:54:53,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 60a6f7bdba888b67b70e7783b1b80897: Waiting for close lock at 1676307293496Running coprocessor pre-close hooks at 1676307293496Disabling compacts and flushes for region at 1676307293496Disabling writes for close at 1676307293497 (+1 ms)Writing region close event to WAL at 1676307293497Running coprocessor post-close hooks at 1676307293501 (+4 ms)Closed at 1676307293501 2023-02-13 16:54:53,501 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 60a6f7bdba888b67b70e7783b1b80897 to self. 2023-02-13 16:54:53,503 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,503 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=60a6f7bdba888b67b70e7783b1b80897, regionState=CLOSED 2023-02-13 16:54:53,503 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307293503"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307293503"}]},"ts":"1676307293503"} 2023-02-13 16:54:53,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-13 16:54:53,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 60a6f7bdba888b67b70e7783b1b80897, server=jenkins-hbase4.apache.org,34717,1676307263076 in 160 msec 2023-02-13 16:54:53,508 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=true 2023-02-13 16:54:53,659 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:54:53,659 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=60a6f7bdba888b67b70e7783b1b80897, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:53,659 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307293659"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307293659"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307293659"}]},"ts":"1676307293659"} 2023-02-13 16:54:53,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 60a6f7bdba888b67b70e7783b1b80897, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:54:53,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 60a6f7bdba888b67b70e7783b1b80897, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:53,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:53,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:53,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,817 INFO [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,817 DEBUG [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/cf1 2023-02-13 16:54:53,817 DEBUG [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/cf1 2023-02-13 16:54:53,818 INFO [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60a6f7bdba888b67b70e7783b1b80897 columnFamilyName cf1 2023-02-13 16:54:53,818 INFO [StoreOpener-60a6f7bdba888b67b70e7783b1b80897-1] regionserver.HStore(310): Store=60a6f7bdba888b67b70e7783b1b80897/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:53,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:54:53,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 60a6f7bdba888b67b70e7783b1b80897; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69998367, jitterRate=0.043056949973106384}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:53,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 60a6f7bdba888b67b70e7783b1b80897: Running coprocessor pre-open hook at 1676307293816Writing region info on filesystem at 1676307293816Initializing all the Stores at 1676307293816Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307293816Cleaning up temporary data from old regions at 1676307293820 (+4 ms)Cleaning up detritus from prior splits at 1676307293820Running coprocessor post-open hooks at 1676307293822 (+2 ms)Region opened successfully at 1676307293822 2023-02-13 16:54:53,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897., pid=92, masterSystemTime=1676307293812 2023-02-13 16:54:53,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:54:53,825 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=60a6f7bdba888b67b70e7783b1b80897, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:54:53,825 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307293825"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307293825"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307293825"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307293825"}]},"ts":"1676307293825"} 2023-02-13 16:54:53,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-13 16:54:53,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 60a6f7bdba888b67b70e7783b1b80897, server=jenkins-hbase4.apache.org,34717,1676307263076 in 165 msec 2023-02-13 16:54:53,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-13 16:54:53,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, REOPEN/MOVE in 486 msec 2023-02-13 16:54:53,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-13 16:54:53,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 503 msec 2023-02-13 16:54:53,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a, ASSIGN}] 2023-02-13 16:54:53,835 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a, ASSIGN 2023-02-13 16:54:53,836 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 16:54:53,986 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:54:53,987 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:54:53,988 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:54:53,988 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:54:53,988 DEBUG [jenkins-hbase4:45273] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-13 16:54:53,989 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=6b6f20b9cb1f3f3857c81c514fc6d98a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:53,989 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307293989"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676307293989"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676307293989"}]},"ts":"1676307293989"} 2023-02-13 16:54:53,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 6b6f20b9cb1f3f3857c81c514fc6d98a, server=jenkins-hbase4.apache.org,41587,1676307262998}] 2023-02-13 16:54:54,145 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:54:54,145 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 16:54:54,148 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51890, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 16:54:54,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a. 2023-02-13 16:54:54,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6b6f20b9cb1f3f3857c81c514fc6d98a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-13 16:54:54,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:54:54,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 6b6f20b9cb1f3f3857c81c514fc6d98a 2023-02-13 16:54:54,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:54,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6b6f20b9cb1f3f3857c81c514fc6d98a 2023-02-13 16:54:54,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6b6f20b9cb1f3f3857c81c514fc6d98a 2023-02-13 16:54:54,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:54,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:54,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:54,361 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-13 16:54:54,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:54,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:54,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:54,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:54,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:54,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:55,981 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:56,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:56,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:56,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:56,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:56,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:56,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:56,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:56,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:56,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:56,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:57,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:58,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:54:59,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:00,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:01,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:02,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:03,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:04,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:05,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:06,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:07,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:08,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:09,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:10,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:11,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:12,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:13,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:14,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:15,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:16,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:16,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:16,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:16,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:16,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:16,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:16,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:16,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:16,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:17,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:18,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:19,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:20,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:21,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:22,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:23,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,386 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 16:55:24,398 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 16:55:24,407 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 16:55:24,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:24,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:25,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:26,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:27,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:28,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:29,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:30,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:31,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:32,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:33,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:34,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:35,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:36,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:37,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:38,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:39,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:40,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:41,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:42,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:43,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:44,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:45,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:46,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:47,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:48,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:49,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:50,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:51,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:52,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 16:55:53,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 16:55:53,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 16:55:53,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:53,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 16:55:54,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 16:55:54,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:54,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:55,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:55,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:55,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:55,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:55,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:55,520 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:55,521 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:55:55,522 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:55:55,522 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:55:55,523 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:55:55,524 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 60a6f7bdba888b67b70e7783b1b80897, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:55:55,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f exec-time=5 msec 2023-02-13 16:55:55,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:55,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 16:55:55,776 INFO [StoreOpener-6b6f20b9cb1f3f3857c81c514fc6d98a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6b6f20b9cb1f3f3857c81c514fc6d98a 2023-02-13 16:55:55,779 DEBUG [StoreOpener-6b6f20b9cb1f3f3857c81c514fc6d98a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/cf1 2023-02-13 16:55:55,779 DEBUG [StoreOpener-6b6f20b9cb1f3f3857c81c514fc6d98a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/cf1 2023-02-13 16:55:55,779 INFO [StoreOpener-6b6f20b9cb1f3f3857c81c514fc6d98a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b6f20b9cb1f3f3857c81c514fc6d98a columnFamilyName cf1 2023-02-13 16:55:55,780 INFO [StoreOpener-6b6f20b9cb1f3f3857c81c514fc6d98a-1] regionserver.HStore(310): Store=6b6f20b9cb1f3f3857c81c514fc6d98a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:55,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6b6f20b9cb1f3f3857c81c514fc6d98a; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73719685, jitterRate=0.09850890934467316}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:55:55,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6b6f20b9cb1f3f3857c81c514fc6d98a: Running coprocessor pre-open hook at 1676307294154Initializing all the Stores at 1676307355775 (+61621 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307355776 (+1 ms)Running coprocessor post-open hooks at 1676307355781 (+5 ms)Region opened successfully at 1676307355781 2023-02-13 16:55:55,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a., pid=94, masterSystemTime=1676307294145 2023-02-13 16:55:55,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a. 2023-02-13 16:55:55,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a. 2023-02-13 16:55:55,785 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=6b6f20b9cb1f3f3857c81c514fc6d98a, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:55,785 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307355784"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676307355784"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676307355784"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676307355784"}]},"ts":"1676307355784"} 2023-02-13 16:55:55,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-13 16:55:55,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 6b6f20b9cb1f3f3857c81c514fc6d98a, server=jenkins-hbase4.apache.org,41587,1676307262998 in 1 mins, 1.795 sec 2023-02-13 16:55:55,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-13 16:55:55,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a, ASSIGN in 1 mins, 1.953 sec 2023-02-13 16:55:55,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.483 sec 2023-02-13 16:55:55,791 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676307293301","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62490,"client":"172.31.14.131:39290","queuetimems":0,"class":"HMaster"} 2023-02-13 16:55:55,792 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 641, skipping over this response of 3 bytes 2023-02-13 16:55:57,772 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up 2023-02-13 16:55:57,774 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:55:57,775 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:57,775 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:57,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:57,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 16:55:57,778 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307357778"}]},"ts":"1676307357778"} 2023-02-13 16:55:57,779 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-13 16:55:57,781 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-13 16:55:57,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a, UNASSIGN}] 2023-02-13 16:55:57,782 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a, UNASSIGN 2023-02-13 16:55:57,782 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6b6f20b9cb1f3f3857c81c514fc6d98a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:57,783 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307357782"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676307357782"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676307357782"}]},"ts":"1676307357782"} 2023-02-13 16:55:57,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 6b6f20b9cb1f3f3857c81c514fc6d98a, server=jenkins-hbase4.apache.org,41587,1676307262998}] 2023-02-13 16:55:57,785 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, UNASSIGN 2023-02-13 16:55:57,786 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=60a6f7bdba888b67b70e7783b1b80897, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:57,786 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307357785"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307357785"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307357785"}]},"ts":"1676307357785"} 2023-02-13 16:55:57,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 60a6f7bdba888b67b70e7783b1b80897, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:55:57,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 16:55:57,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6b6f20b9cb1f3f3857c81c514fc6d98a 2023-02-13 16:55:57,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6b6f20b9cb1f3f3857c81c514fc6d98a, disabling compactions & flushes 2023-02-13 16:55:57,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a. 2023-02-13 16:55:57,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a. 2023-02-13 16:55:57,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a. after waiting 0 ms 2023-02-13 16:55:57,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a. 2023-02-13 16:55:57,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:55:57,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a. 2023-02-13 16:55:57,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6b6f20b9cb1f3f3857c81c514fc6d98a: Waiting for close lock at 1676307357936Running coprocessor pre-close hooks at 1676307357936Disabling compacts and flushes for region at 1676307357936Disabling writes for close at 1676307357937 (+1 ms)Writing region close event to WAL at 1676307357937Running coprocessor post-close hooks at 1676307357937Closed at 1676307357938 (+1 ms) 2023-02-13 16:55:57,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:55:57,939 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 60a6f7bdba888b67b70e7783b1b80897, disabling compactions & flushes 2023-02-13 16:55:57,939 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:55:57,939 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:55:57,939 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. after waiting 0 ms 2023-02-13 16:55:57,939 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:55:57,942 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6b6f20b9cb1f3f3857c81c514fc6d98a 2023-02-13 16:55:57,943 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6b6f20b9cb1f3f3857c81c514fc6d98a, regionState=CLOSED 2023-02-13 16:55:57,943 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307357943"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676307357943"}]},"ts":"1676307357943"} 2023-02-13 16:55:57,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 16:55:57,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:55:57,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897. 2023-02-13 16:55:57,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 60a6f7bdba888b67b70e7783b1b80897: Waiting for close lock at 1676307357938Running coprocessor pre-close hooks at 1676307357938Disabling compacts and flushes for region at 1676307357938Disabling writes for close at 1676307357939 (+1 ms)Writing region close event to WAL at 1676307357939Running coprocessor post-close hooks at 1676307357948 (+9 ms)Closed at 1676307357948 2023-02-13 16:55:57,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-13 16:55:57,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 6b6f20b9cb1f3f3857c81c514fc6d98a, server=jenkins-hbase4.apache.org,41587,1676307262998 in 162 msec 2023-02-13 16:55:57,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:55:57,951 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a, UNASSIGN in 169 msec 2023-02-13 16:55:57,952 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=60a6f7bdba888b67b70e7783b1b80897, regionState=CLOSED 2023-02-13 16:55:57,952 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676307357952"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307357952"}]},"ts":"1676307357952"} 2023-02-13 16:55:57,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-13 16:55:57,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 60a6f7bdba888b67b70e7783b1b80897, server=jenkins-hbase4.apache.org,34717,1676307263076 in 167 msec 2023-02-13 16:55:57,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-13 16:55:57,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=60a6f7bdba888b67b70e7783b1b80897, UNASSIGN in 177 msec 2023-02-13 16:55:57,961 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307357961"}]},"ts":"1676307357961"} 2023-02-13 16:55:57,962 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-13 16:55:57,964 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-13 16:55:57,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 190 msec 2023-02-13 16:55:58,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 16:55:58,080 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-13 16:55:58,081 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:58,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:58,084 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:58,084 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:58,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 16:55:58,104 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:55:58,107 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/recovered.edits] 2023-02-13 16:55:58,112 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/recovered.edits/7.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897/recovered.edits/7.seqid 2023-02-13 16:55:58,117 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/60a6f7bdba888b67b70e7783b1b80897 2023-02-13 16:55:58,117 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 16:55:58,120 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:58,134 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-13 16:55:58,144 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-13 16:55:58,145 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:58,145 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-13 16:55:58,146 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307358145"}]},"ts":"9223372036854775807"} 2023-02-13 16:55:58,146 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307358145"}]},"ts":"9223372036854775807"} 2023-02-13 16:55:58,149 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-13 16:55:58,149 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 60a6f7bdba888b67b70e7783b1b80897, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676307292793.60a6f7bdba888b67b70e7783b1b80897.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6b6f20b9cb1f3f3857c81c514fc6d98a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676307292793_0001.6b6f20b9cb1f3f3857c81c514fc6d98a.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-13 16:55:58,149 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-13 16:55:58,149 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307358149"}]},"ts":"9223372036854775807"} 2023-02-13 16:55:58,152 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-13 16:55:58,154 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:55:58,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 73 msec 2023-02-13 16:55:58,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 16:55:58,188 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-13 16:55:58,204 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=345 (was 405), OpenFileDescriptor=656 (was 854), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=117 (was 210), ProcessCount=168 (was 168), AvailableMemoryMB=1105 (was 1257) 2023-02-13 16:55:58,219 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=346, OpenFileDescriptor=656, MaxFileDescriptor=60000, SystemLoadAverage=117, ProcessCount=168, AvailableMemoryMB=1104 2023-02-13 16:55:58,219 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:55:58,223 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:55:58,223 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:55:58,224 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:55:58,226 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:55:58,226 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:55:58,228 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,229 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1 empty. 2023-02-13 16:55:58,229 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,229 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 16:55:58,241 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-13 16:55:58,242 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => a4faf7341bc5f1c5c43d03ec50c1d9a1, NAME => 'testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:55:58,249 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:58,249 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing a4faf7341bc5f1c5c43d03ec50c1d9a1, disabling compactions & flushes 2023-02-13 16:55:58,249 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,249 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,250 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. after waiting 0 ms 2023-02-13 16:55:58,250 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,250 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,250 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: Waiting for close lock at 1676307358249Disabling compacts and flushes for region at 1676307358249Disabling writes for close at 1676307358250 (+1 ms)Writing region close event to WAL at 1676307358250Closed at 1676307358250 2023-02-13 16:55:58,251 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:55:58,252 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676307358252"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307358252"}]},"ts":"1676307358252"} 2023-02-13 16:55:58,253 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:55:58,254 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:55:58,254 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307358254"}]},"ts":"1676307358254"} 2023-02-13 16:55:58,255 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-13 16:55:58,258 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:55:58,259 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:55:58,259 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:55:58,259 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:55:58,259 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:55:58,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=a4faf7341bc5f1c5c43d03ec50c1d9a1, ASSIGN}] 2023-02-13 16:55:58,261 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=a4faf7341bc5f1c5c43d03ec50c1d9a1, ASSIGN 2023-02-13 16:55:58,261 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=a4faf7341bc5f1c5c43d03ec50c1d9a1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:55:58,411 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:55:58,412 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=a4faf7341bc5f1c5c43d03ec50c1d9a1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,413 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676307358412"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307358412"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307358412"}]},"ts":"1676307358412"} 2023-02-13 16:55:58,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure a4faf7341bc5f1c5c43d03ec50c1d9a1, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:55:58,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a4faf7341bc5f1c5c43d03ec50c1d9a1, NAME => 'testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:55:58,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:55:58,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:58,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,570 INFO [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,572 DEBUG [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1 2023-02-13 16:55:58,572 DEBUG [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1 2023-02-13 16:55:58,572 INFO [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a4faf7341bc5f1c5c43d03ec50c1d9a1 columnFamilyName cf1 2023-02-13 16:55:58,573 INFO [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] regionserver.HStore(310): Store=a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:58,573 INFO [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,574 DEBUG [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2 2023-02-13 16:55:58,574 DEBUG [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2 2023-02-13 16:55:58,574 INFO [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a4faf7341bc5f1c5c43d03ec50c1d9a1 columnFamilyName cf2 2023-02-13 16:55:58,575 INFO [StoreOpener-a4faf7341bc5f1c5c43d03ec50c1d9a1-1] regionserver.HStore(310): Store=a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:58,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:58,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:58,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:55:58,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a4faf7341bc5f1c5c43d03ec50c1d9a1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66388710, jitterRate=-0.010731130838394165}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:58,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: Running coprocessor pre-open hook at 1676307358569Writing region info on filesystem at 1676307358569Initializing all the Stores at 1676307358570 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307358570Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307358570Cleaning up temporary data from old regions at 1676307358576 (+6 ms)Cleaning up detritus from prior splits at 1676307358576Running coprocessor post-open hooks at 1676307358581 (+5 ms)Region opened successfully at 1676307358581 2023-02-13 16:55:58,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1., pid=104, masterSystemTime=1676307358566 2023-02-13 16:55:58,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,584 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=a4faf7341bc5f1c5c43d03ec50c1d9a1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,584 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676307358583"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307358583"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307358583"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307358583"}]},"ts":"1676307358583"} 2023-02-13 16:55:58,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-13 16:55:58,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure a4faf7341bc5f1c5c43d03ec50c1d9a1, server=jenkins-hbase4.apache.org,34717,1676307263076 in 171 msec 2023-02-13 16:55:58,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-13 16:55:58,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=a4faf7341bc5f1c5c43d03ec50c1d9a1, ASSIGN in 327 msec 2023-02-13 16:55:58,589 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:55:58,589 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307358589"}]},"ts":"1676307358589"} 2023-02-13 16:55:58,590 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-13 16:55:58,592 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:55:58,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 368 msec 2023-02-13 16:55:58,732 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:58,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 16:55:58,733 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 16:55:58,733 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:58,733 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 16:55:58,733 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 16:55:58,734 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,734 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,736 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,736 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,736 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,736 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,736 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,736 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:58,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:58,737 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:58,737 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:58,737 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:58,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:58,737 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:58,738 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:58,738 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:58,738 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:58,738 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:58,738 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:58,738 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:58,738 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:58,738 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:55:58,738 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:58,738 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,738 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. started... 2023-02-13 16:55:58,739 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4faf7341bc5f1c5c43d03ec50c1d9a1 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 16:55:58,739 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,740 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,740 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,740 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:58,740 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,740 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:58,748 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/f1fa0c8a98b3435693301cad190f132f 2023-02-13 16:55:58,759 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/ef015aca8bfa476b9c4a78efeac4b9f3 2023-02-13 16:55:58,762 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/f1fa0c8a98b3435693301cad190f132f as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/f1fa0c8a98b3435693301cad190f132f 2023-02-13 16:55:58,766 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/f1fa0c8a98b3435693301cad190f132f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:55:58,766 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/ef015aca8bfa476b9c4a78efeac4b9f3 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/ef015aca8bfa476b9c4a78efeac4b9f3 2023-02-13 16:55:58,770 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/ef015aca8bfa476b9c4a78efeac4b9f3, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:55:58,770 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a4faf7341bc5f1c5c43d03ec50c1d9a1 in 31ms, sequenceid=6, compaction requested=false 2023-02-13 16:55:58,770 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-13 16:55:58,771 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: 2023-02-13 16:55:58,771 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,771 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:55:58,771 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:55:58,771 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,771 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:58,771 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:58,772 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,772 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:58,773 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,773 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:58,775 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 16:55:58,775 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10362e79[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:58,775 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 16:55:58,775 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,776 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,776 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,776 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:58,777 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,777 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:55:58,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:55:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:55:58,778 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:58,778 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:58,778 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:58,778 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:58,778 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:58,778 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:58,779 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,779 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:58,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:58,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:58,779 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:58,780 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:58,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:58,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:58,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:58,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:55:58,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 16:55:58,782 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 16:55:58,782 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:58,782 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:58,782 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 16:55:58,782 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:58,784 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:58,784 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,784 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:58,784 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,784 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,785 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,785 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:58,786 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,787 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,787 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,790 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 16:55:58,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,790 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:58,790 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:58,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,790 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,791 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 16:55:58,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,791 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,791 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:58,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,791 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,791 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,791 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,791 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,791 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,891 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:58,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:58,903 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:58,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:58,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 16:55:58,904 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 16:55:58,905 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:58,905 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 16:55:58,905 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 16:55:58,905 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,905 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,906 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,906 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,906 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,906 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,907 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,907 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:58,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:58,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:58,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,908 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:58,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:58,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,908 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:58,908 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:58,908 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:58,908 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:58,908 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:58,908 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:58,908 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:58,908 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:58,909 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:55:58,909 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:58,909 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,909 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. started... 2023-02-13 16:55:58,909 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4faf7341bc5f1c5c43d03ec50c1d9a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:55:58,909 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,914 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,914 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,914 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:58,914 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,914 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:58,917 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/43f966e1a85242f48e638287635eebe8 2023-02-13 16:55:58,928 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/5798370477fc4ef68c7fdad321966021 2023-02-13 16:55:58,932 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/43f966e1a85242f48e638287635eebe8 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/43f966e1a85242f48e638287635eebe8 2023-02-13 16:55:58,936 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/43f966e1a85242f48e638287635eebe8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:55:58,936 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/5798370477fc4ef68c7fdad321966021 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/5798370477fc4ef68c7fdad321966021 2023-02-13 16:55:58,940 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/5798370477fc4ef68c7fdad321966021, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:55:58,941 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4faf7341bc5f1c5c43d03ec50c1d9a1 in 32ms, sequenceid=19, compaction requested=false 2023-02-13 16:55:58,941 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: 2023-02-13 16:55:58,941 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:58,941 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:55:58,941 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:55:58,941 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,941 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:58,941 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:58,943 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,943 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:58,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:58,943 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:58,943 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:58,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:58,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:58,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 16:55:58,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b73dedb[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:58,946 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 16:55:58,946 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:58,947 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:58,947 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,947 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:58,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,947 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:58,947 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:58,947 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:55:58,947 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:58,947 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:55:58,947 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:58,947 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:55:58,950 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,950 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:58,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:58,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:58,950 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:58,950 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:58,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:58,951 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:58,951 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:58,951 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:58,951 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:58,951 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:58,951 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:58,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:58,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:58,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:55:58,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 16:55:58,953 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 16:55:58,954 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:58,954 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:58,954 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 16:55:58,954 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:58,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:58,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:58,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,956 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:58,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,956 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,957 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:58,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:58,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,958 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:58,958 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:58,959 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,961 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,961 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,961 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,961 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,961 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,962 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:58,961 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:58,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 16:55:58,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:58,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,962 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:58,962 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:58,962 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:58,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,962 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 16:55:58,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,962 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:58,962 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:58,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:58,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:58,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:58,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:58,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:58,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:58,963 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,063 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:59,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:59,075 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:59,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:59,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 16:55:59,077 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 16:55:59,077 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,077 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 16:55:59,077 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 16:55:59,077 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,077 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,079 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,079 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,079 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,079 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,080 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,080 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:59,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:55:59,082 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:59,082 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,082 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. started... 2023-02-13 16:55:59,082 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4faf7341bc5f1c5c43d03ec50c1d9a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:55:59,083 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,083 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,083 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,083 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,083 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,083 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,092 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/34a4bbdf8e2f4193a4159f0f6280046d 2023-02-13 16:55:59,104 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/bd34b6c5fd354948a24af2997f50c945 2023-02-13 16:55:59,108 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/34a4bbdf8e2f4193a4159f0f6280046d as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/34a4bbdf8e2f4193a4159f0f6280046d 2023-02-13 16:55:59,111 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/34a4bbdf8e2f4193a4159f0f6280046d, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:55:59,112 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/bd34b6c5fd354948a24af2997f50c945 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/bd34b6c5fd354948a24af2997f50c945 2023-02-13 16:55:59,115 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/bd34b6c5fd354948a24af2997f50c945, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:55:59,116 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4faf7341bc5f1c5c43d03ec50c1d9a1 in 34ms, sequenceid=32, compaction requested=true 2023-02-13 16:55:59,116 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: 2023-02-13 16:55:59,116 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,116 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:55:59,116 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:55:59,116 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,116 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,116 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,117 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,117 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,118 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,118 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 16:55:59,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2740fa57[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:59,119 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 16:55:59,119 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,120 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,121 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,121 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,121 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,121 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:59,121 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,121 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,121 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,121 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,121 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,121 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,121 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:55:59,121 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:55:59,121 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:55:59,123 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,123 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,123 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,124 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:55:59,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 16:55:59,127 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 16:55:59,127 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:59,127 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:59,127 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 16:55:59,127 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:59,128 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,128 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,128 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,128 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,128 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,128 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,128 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,129 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,129 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,130 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,131 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,131 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,132 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,137 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,137 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,137 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,138 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,138 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 16:55:59,138 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:59,137 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,137 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,137 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,138 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 16:55:59,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,138 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:59,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,239 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:59,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:59,251 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:59,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:59,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 16:55:59,252 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 16:55:59,252 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,252 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 16:55:59,252 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 16:55:59,253 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,253 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,254 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,254 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,254 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,254 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,254 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,254 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:59,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,255 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,255 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,256 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,256 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,256 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:55:59,256 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:59,256 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,256 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. started... 2023-02-13 16:55:59,257 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4faf7341bc5f1c5c43d03ec50c1d9a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:55:59,257 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,257 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,258 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,258 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,258 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,258 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,266 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/12310e37f36a4ec884ffd7f9e590e129 2023-02-13 16:55:59,282 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/706b8564ea8f4bcda8037c8499d8c4b5 2023-02-13 16:55:59,286 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/12310e37f36a4ec884ffd7f9e590e129 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/12310e37f36a4ec884ffd7f9e590e129 2023-02-13 16:55:59,290 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/12310e37f36a4ec884ffd7f9e590e129, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:55:59,291 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/706b8564ea8f4bcda8037c8499d8c4b5 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/706b8564ea8f4bcda8037c8499d8c4b5 2023-02-13 16:55:59,294 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/706b8564ea8f4bcda8037c8499d8c4b5, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:55:59,295 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4faf7341bc5f1c5c43d03ec50c1d9a1 in 39ms, sequenceid=45, compaction requested=true 2023-02-13 16:55:59,295 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: 2023-02-13 16:55:59,295 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:55:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:55:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,298 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,298 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,298 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,298 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 16:55:59,305 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@635fb6f1[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:59,305 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 16:55:59,305 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,306 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,306 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:59,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,306 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,306 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,306 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:55:59,306 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:55:59,308 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,308 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,308 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,308 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,308 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,308 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,309 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,309 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,309 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:55:59,311 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,311 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,311 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,311 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:55:59,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 16:55:59,314 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 16:55:59,314 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:59,314 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:59,314 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 16:55:59,314 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:59,316 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,317 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,317 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,317 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,318 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,318 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,320 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,320 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,320 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,322 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,322 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,322 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,322 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,322 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,323 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,323 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,323 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:59,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,323 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,324 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,324 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,324 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 16:55:59,324 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:59,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,424 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:59,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:59,435 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:59,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:59,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 16:55:59,436 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 16:55:59,436 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,436 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 16:55:59,436 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 16:55:59,437 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,437 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,438 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,438 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,438 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,438 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,439 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,439 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,440 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,440 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,440 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,440 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,440 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,440 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,440 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:59,440 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,440 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. started... 2023-02-13 16:55:59,441 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4faf7341bc5f1c5c43d03ec50c1d9a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:55:59,442 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,442 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,442 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,442 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,442 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,442 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,449 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/9a629430fb2b4318b44109918783493a 2023-02-13 16:55:59,459 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/2985ace6228142b7bef38c4a1eb6bd93 2023-02-13 16:55:59,462 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/9a629430fb2b4318b44109918783493a as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/9a629430fb2b4318b44109918783493a 2023-02-13 16:55:59,466 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/9a629430fb2b4318b44109918783493a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:55:59,467 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/2985ace6228142b7bef38c4a1eb6bd93 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/2985ace6228142b7bef38c4a1eb6bd93 2023-02-13 16:55:59,470 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/2985ace6228142b7bef38c4a1eb6bd93, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:55:59,471 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4faf7341bc5f1c5c43d03ec50c1d9a1 in 30ms, sequenceid=58, compaction requested=true 2023-02-13 16:55:59,471 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: 2023-02-13 16:55:59,471 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,471 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:55:59,471 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:55:59,471 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,471 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,471 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,473 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,473 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,474 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,474 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 16:55:59,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42bb1905[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:59,475 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 16:55:59,475 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,476 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,476 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,476 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,477 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,477 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,477 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,477 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:59,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,477 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,477 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,477 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,477 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:55:59,477 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,477 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:55:59,477 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:55:59,481 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,481 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,481 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,481 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,485 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,485 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,485 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,485 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,485 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,485 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:55:59,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 16:55:59,488 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 16:55:59,488 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:59,488 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:59,488 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 16:55:59,488 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:59,489 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,489 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,489 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,489 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,489 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,489 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,489 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,490 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,490 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,491 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,492 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,492 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,493 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,495 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 16:55:59,495 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,495 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:59,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,495 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,496 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,496 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,496 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,496 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,496 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,496 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,496 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,596 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:59,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:59,606 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:59,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:59,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 16:55:59,608 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 16:55:59,608 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,608 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 16:55:59,608 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 16:55:59,608 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,608 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,610 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,610 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,610 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,610 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 16:55:59,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,611 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,611 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 16:55:59,611 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,611 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 16:55:59,611 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,611 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,612 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,612 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,612 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,611 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:59,612 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,612 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,612 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 16:55:59,612 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:55:59,612 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:59,612 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,612 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. started... 2023-02-13 16:55:59,612 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4faf7341bc5f1c5c43d03ec50c1d9a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:55:59,614 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,614 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,615 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,615 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,615 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,615 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,620 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/6abc19fa00614ba98b32f45fd889fc23 2023-02-13 16:55:59,630 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/6df0f5095b5946d2b7e0572a34f87bc9 2023-02-13 16:55:59,633 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/6abc19fa00614ba98b32f45fd889fc23 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/6abc19fa00614ba98b32f45fd889fc23 2023-02-13 16:55:59,638 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/6abc19fa00614ba98b32f45fd889fc23, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:55:59,638 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/6df0f5095b5946d2b7e0572a34f87bc9 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/6df0f5095b5946d2b7e0572a34f87bc9 2023-02-13 16:55:59,642 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/6df0f5095b5946d2b7e0572a34f87bc9, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:55:59,642 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4faf7341bc5f1c5c43d03ec50c1d9a1 in 30ms, sequenceid=71, compaction requested=true 2023-02-13 16:55:59,642 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: 2023-02-13 16:55:59,642 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,643 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:55:59,643 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:55:59,643 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,643 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 16:55:59,643 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 16:55:59,644 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,644 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,644 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,644 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:59,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 16:55:59,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7fb828f[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:59,646 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 16:55:59,646 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,647 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:59,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,647 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,648 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,647 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,648 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:55:59,648 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:55:59,648 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 16:55:59,648 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 16:55:59,648 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:55:59,649 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,649 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,649 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,650 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,650 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,650 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,650 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 16:55:59,651 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,651 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,651 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 16:55:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:55:59,653 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 16:55:59,653 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 16:55:59,653 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:59,653 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:59,653 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 16:55:59,653 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:59,655 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:59,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,655 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,655 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,656 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:59,656 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:59,658 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 16:55:59,658 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,658 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,660 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:59,660 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 16:55:59,660 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:59,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:59,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 16:55:59,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:59,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 16:55:59,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:55:59,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:59,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 16:55:59,661 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 16:55:59,692 ERROR [BP-2104803542-172.31.14.131-1676307258605 heartbeating to localhost/127.0.0.1:33763] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104803542-172.31.14.131-1676307258605 (Datanode Uuid a3585e58-faa4-4a64-84b5-04c6ddf75562) service to localhost/127.0.0.1:33763 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:55:59,761 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:59,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:59,768 INFO [Listener at localhost/38547] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,769 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.StoreUtils(129): cannot split hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/43f966e1a85242f48e638287635eebe8 because midkey is the same as first or last row 2023-02-13 16:55:59,769 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.StoreUtils(129): cannot split hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/5798370477fc4ef68c7fdad321966021 because midkey is the same as first or last row 2023-02-13 16:55:59,770 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegion(2744): Flushing a4faf7341bc5f1c5c43d03ec50c1d9a1 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 16:55:59,779 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/f8f0dc51e43241e896c41e44fec8308f 2023-02-13 16:55:59,788 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/c9b061133ab44d989db8ec2e67087ff3 2023-02-13 16:55:59,792 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf1/f8f0dc51e43241e896c41e44fec8308f as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/f8f0dc51e43241e896c41e44fec8308f 2023-02-13 16:55:59,795 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/f8f0dc51e43241e896c41e44fec8308f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:55:59,795 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/.tmp/cf2/c9b061133ab44d989db8ec2e67087ff3 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/c9b061133ab44d989db8ec2e67087ff3 2023-02-13 16:55:59,798 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/c9b061133ab44d989db8ec2e67087ff3, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:55:59,799 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a4faf7341bc5f1c5c43d03ec50c1d9a1 in 29ms, sequenceid=82, compaction requested=true 2023-02-13 16:55:59,800 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.HRegion(2445): Flush status journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: 2023-02-13 16:55:59,800 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.StoreUtils(129): cannot split hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/43f966e1a85242f48e638287635eebe8 because midkey is the same as first or last row 2023-02-13 16:55:59,800 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34717] regionserver.StoreUtils(129): cannot split hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/5798370477fc4ef68c7fdad321966021 because midkey is the same as first or last row 2023-02-13 16:55:59,800 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,800 DEBUG [Listener at localhost/38547] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-13 16:55:59,800 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:55:59,802 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-13 16:55:59,802 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-13 16:55:59,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:55:59,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 16:55:59,805 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307359805"}]},"ts":"1676307359805"} 2023-02-13 16:55:59,806 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-13 16:55:59,808 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-13 16:55:59,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=a4faf7341bc5f1c5c43d03ec50c1d9a1, UNASSIGN}] 2023-02-13 16:55:59,809 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=a4faf7341bc5f1c5c43d03ec50c1d9a1, UNASSIGN 2023-02-13 16:55:59,810 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=a4faf7341bc5f1c5c43d03ec50c1d9a1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:55:59,810 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676307359810"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307359810"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307359810"}]},"ts":"1676307359810"} 2023-02-13 16:55:59,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure a4faf7341bc5f1c5c43d03ec50c1d9a1, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:55:59,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 16:55:59,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:59,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a4faf7341bc5f1c5c43d03ec50c1d9a1, disabling compactions & flushes 2023-02-13 16:55:59,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. after waiting 0 ms 2023-02-13 16:55:59,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 16:55:59,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:55:59,968 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1. 2023-02-13 16:55:59,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a4faf7341bc5f1c5c43d03ec50c1d9a1: Waiting for close lock at 1676307359963Running coprocessor pre-close hooks at 1676307359963Disabling compacts and flushes for region at 1676307359963Disabling writes for close at 1676307359963Writing region close event to WAL at 1676307359964 (+1 ms)Running coprocessor post-close hooks at 1676307359968 (+4 ms)Closed at 1676307359968 2023-02-13 16:55:59,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:55:59,969 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=a4faf7341bc5f1c5c43d03ec50c1d9a1, regionState=CLOSED 2023-02-13 16:55:59,969 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676307359969"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307359969"}]},"ts":"1676307359969"} 2023-02-13 16:55:59,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-13 16:55:59,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure a4faf7341bc5f1c5c43d03ec50c1d9a1, server=jenkins-hbase4.apache.org,34717,1676307263076 in 159 msec 2023-02-13 16:55:59,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-13 16:55:59,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=a4faf7341bc5f1c5c43d03ec50c1d9a1, UNASSIGN in 163 msec 2023-02-13 16:55:59,973 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307359973"}]},"ts":"1676307359973"} 2023-02-13 16:55:59,974 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-13 16:55:59,975 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-13 16:55:59,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 174 msec 2023-02-13 16:56:00,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 16:56:00,107 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-13 16:56:00,107 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-13 16:56:00,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:00,110 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:00,110 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:00,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 16:56:00,113 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:56:00,115 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/recovered.edits] 2023-02-13 16:56:00,118 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/12310e37f36a4ec884ffd7f9e590e129 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/12310e37f36a4ec884ffd7f9e590e129 2023-02-13 16:56:00,119 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/34a4bbdf8e2f4193a4159f0f6280046d to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/34a4bbdf8e2f4193a4159f0f6280046d 2023-02-13 16:56:00,120 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/43f966e1a85242f48e638287635eebe8 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/43f966e1a85242f48e638287635eebe8 2023-02-13 16:56:00,121 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/6abc19fa00614ba98b32f45fd889fc23 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/6abc19fa00614ba98b32f45fd889fc23 2023-02-13 16:56:00,122 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/9a629430fb2b4318b44109918783493a to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/9a629430fb2b4318b44109918783493a 2023-02-13 16:56:00,123 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/f1fa0c8a98b3435693301cad190f132f to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/f1fa0c8a98b3435693301cad190f132f 2023-02-13 16:56:00,124 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/f8f0dc51e43241e896c41e44fec8308f to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf1/f8f0dc51e43241e896c41e44fec8308f 2023-02-13 16:56:00,127 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/2985ace6228142b7bef38c4a1eb6bd93 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/2985ace6228142b7bef38c4a1eb6bd93 2023-02-13 16:56:00,128 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/5798370477fc4ef68c7fdad321966021 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/5798370477fc4ef68c7fdad321966021 2023-02-13 16:56:00,129 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/6df0f5095b5946d2b7e0572a34f87bc9 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/6df0f5095b5946d2b7e0572a34f87bc9 2023-02-13 16:56:00,130 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/706b8564ea8f4bcda8037c8499d8c4b5 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/706b8564ea8f4bcda8037c8499d8c4b5 2023-02-13 16:56:00,131 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/bd34b6c5fd354948a24af2997f50c945 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/bd34b6c5fd354948a24af2997f50c945 2023-02-13 16:56:00,132 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/c9b061133ab44d989db8ec2e67087ff3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/c9b061133ab44d989db8ec2e67087ff3 2023-02-13 16:56:00,133 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/ef015aca8bfa476b9c4a78efeac4b9f3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/cf2/ef015aca8bfa476b9c4a78efeac4b9f3 2023-02-13 16:56:00,134 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/recovered.edits/85.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1/recovered.edits/85.seqid 2023-02-13 16:56:00,135 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testInvalidSplitKey/a4faf7341bc5f1c5c43d03ec50c1d9a1 2023-02-13 16:56:00,135 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 16:56:00,137 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:00,138 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-13 16:56:00,141 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-13 16:56:00,142 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:00,142 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-13 16:56:00,142 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307360142"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:00,143 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 16:56:00,144 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a4faf7341bc5f1c5c43d03ec50c1d9a1, NAME => 'testInvalidSplitKey,,1676307358224.a4faf7341bc5f1c5c43d03ec50c1d9a1.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 16:56:00,144 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-13 16:56:00,144 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307360144"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:00,145 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-13 16:56:00,146 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:00,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 38 msec 2023-02-13 16:56:00,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 16:56:00,211 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-13 16:56:00,225 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=364 (was 346) - Thread LEAK? -, OpenFileDescriptor=735 (was 656) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=117 (was 117), ProcessCount=168 (was 168), AvailableMemoryMB=1100 (was 1104) 2023-02-13 16:56:00,237 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=364, OpenFileDescriptor=735, MaxFileDescriptor=60000, SystemLoadAverage=117, ProcessCount=168, AvailableMemoryMB=1099 2023-02-13 16:56:00,237 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:56:00,239 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:56:00,240 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:56:00,241 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:00,242 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:56:00,243 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:56:00,244 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,245 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3 empty. 2023-02-13 16:56:00,245 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,245 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 16:56:00,257 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 16:56:00,257 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 03971683bd55396998b5a281a250cba3, NAME => 'testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:56:00,266 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:00,266 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 03971683bd55396998b5a281a250cba3, disabling compactions & flushes 2023-02-13 16:56:00,266 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,266 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,266 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. after waiting 0 ms 2023-02-13 16:56:00,266 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,266 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,266 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 03971683bd55396998b5a281a250cba3: Waiting for close lock at 1676307360266Disabling compacts and flushes for region at 1676307360266Disabling writes for close at 1676307360266Writing region close event to WAL at 1676307360266Closed at 1676307360266 2023-02-13 16:56:00,268 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:56:00,269 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307360269"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307360269"}]},"ts":"1676307360269"} 2023-02-13 16:56:00,270 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:56:00,270 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:56:00,271 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307360270"}]},"ts":"1676307360270"} 2023-02-13 16:56:00,271 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 16:56:00,275 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:56:00,276 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:56:00,276 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:56:00,276 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:56:00,276 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:56:00,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, ASSIGN}] 2023-02-13 16:56:00,277 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, ASSIGN 2023-02-13 16:56:00,278 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41587,1676307262998; forceNewPlan=false, retain=false 2023-02-13 16:56:00,428 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:56:00,429 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=03971683bd55396998b5a281a250cba3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,429 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307360429"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307360429"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307360429"}]},"ts":"1676307360429"} 2023-02-13 16:56:00,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998}] 2023-02-13 16:56:00,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 03971683bd55396998b5a281a250cba3, NAME => 'testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:56:00,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:56:00,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:00,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,587 INFO [StoreOpener-03971683bd55396998b5a281a250cba3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,588 DEBUG [StoreOpener-03971683bd55396998b5a281a250cba3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1 2023-02-13 16:56:00,588 DEBUG [StoreOpener-03971683bd55396998b5a281a250cba3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1 2023-02-13 16:56:00,588 INFO [StoreOpener-03971683bd55396998b5a281a250cba3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 03971683bd55396998b5a281a250cba3 columnFamilyName cf1 2023-02-13 16:56:00,589 INFO [StoreOpener-03971683bd55396998b5a281a250cba3-1] regionserver.HStore(310): Store=03971683bd55396998b5a281a250cba3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:00,589 INFO [StoreOpener-03971683bd55396998b5a281a250cba3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,590 DEBUG [StoreOpener-03971683bd55396998b5a281a250cba3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2 2023-02-13 16:56:00,590 DEBUG [StoreOpener-03971683bd55396998b5a281a250cba3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2 2023-02-13 16:56:00,590 INFO [StoreOpener-03971683bd55396998b5a281a250cba3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 03971683bd55396998b5a281a250cba3 columnFamilyName cf2 2023-02-13 16:56:00,590 INFO [StoreOpener-03971683bd55396998b5a281a250cba3-1] regionserver.HStore(310): Store=03971683bd55396998b5a281a250cba3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:00,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:00,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:00,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:56:00,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 03971683bd55396998b5a281a250cba3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74607032, jitterRate=0.11173141002655029}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:00,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 03971683bd55396998b5a281a250cba3: Running coprocessor pre-open hook at 1676307360586Writing region info on filesystem at 1676307360586Initializing all the Stores at 1676307360586Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307360586Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307360586Cleaning up temporary data from old regions at 1676307360592 (+6 ms)Cleaning up detritus from prior splits at 1676307360592Running coprocessor post-open hooks at 1676307360595 (+3 ms)Region opened successfully at 1676307360595 2023-02-13 16:56:00,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3., pid=111, masterSystemTime=1676307360582 2023-02-13 16:56:00,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,598 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=03971683bd55396998b5a281a250cba3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,598 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307360598"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307360598"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307360598"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307360598"}]},"ts":"1676307360598"} 2023-02-13 16:56:00,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-13 16:56:00,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 in 168 msec 2023-02-13 16:56:00,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-13 16:56:00,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, ASSIGN in 324 msec 2023-02-13 16:56:00,602 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:56:00,602 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307360602"}]},"ts":"1676307360602"} 2023-02-13 16:56:00,603 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 16:56:00,605 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:56:00,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 365 msec 2023-02-13 16:56:00,743 DEBUG [Listener at localhost/38547] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:56:00,748 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32860, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:56:00,752 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:00,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 16:56:00,754 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:00,754 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,754 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 16:56:00,754 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:56:00,754 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,754 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,756 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,756 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,756 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,756 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,756 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,756 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:00,757 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:00,757 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,757 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:00,757 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,757 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,757 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:00,758 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:00,758 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:00,758 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:00,757 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:00,758 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:00,758 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:00,758 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:00,758 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:00,758 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,758 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. started... 2023-02-13 16:56:00,759 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 03971683bd55396998b5a281a250cba3 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 16:56:00,760 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:00,767 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1 2023-02-13 16:56:00,782 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/4c0969c0db6b4e8fb10dab98eb864d2e 2023-02-13 16:56:00,786 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1 2023-02-13 16:56:00,789 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:56:00,790 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/4c0969c0db6b4e8fb10dab98eb864d2e as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e 2023-02-13 16:56:00,794 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:56:00,794 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 03971683bd55396998b5a281a250cba3 in 35ms, sequenceid=6, compaction requested=false 2023-02-13 16:56:00,794 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:00,795 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 03971683bd55396998b5a281a250cba3: 2023-02-13 16:56:00,795 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,795 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:00,795 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:00,795 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,795 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:00,795 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:00,796 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,796 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:00,797 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,797 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:00,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 16:56:00,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27495265[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:00,798 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:56:00,799 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,799 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,800 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,799 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,800 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:00,800 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:00,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,800 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,800 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:00,800 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:00,800 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:00,800 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:00,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,800 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:00,800 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:00,800 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:00,800 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:00,805 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:00,805 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:00,805 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:00,806 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:00,806 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:00,806 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:00,806 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:00,806 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:00,806 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,806 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:00,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:00,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:00,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:00,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:00,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:00,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,41587,1676307262998': 2023-02-13 16:56:00,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:56:00,809 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 16:56:00,809 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:00,809 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:00,809 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 16:56:00,809 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:00,810 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,810 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,810 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,810 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:00,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:00,810 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,810 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,811 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:00,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,811 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,811 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:00,812 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:00,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,813 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,813 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,814 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,816 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:00,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,816 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:00,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,816 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:00,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,817 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:00,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,817 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:00,917 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:00,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:00,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:00,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:00,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 16:56:00,930 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:00,930 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,930 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 16:56:00,930 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:56:00,931 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,931 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,933 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,933 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,933 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,933 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,933 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,933 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:00,934 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:00,935 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,935 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:00,935 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:00,935 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:00,935 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:00,935 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,935 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. started... 2023-02-13 16:56:00,936 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 03971683bd55396998b5a281a250cba3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:00,936 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,937 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,937 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,937 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:00,937 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,937 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:00,944 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/64fe6bc523d7443495e81ec2e08af459 2023-02-13 16:56:00,954 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/5cfd5794d1f848b2ba61abf4dd522a69 2023-02-13 16:56:00,957 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/64fe6bc523d7443495e81ec2e08af459 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459 2023-02-13 16:56:00,960 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:56:00,961 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/5cfd5794d1f848b2ba61abf4dd522a69 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69 2023-02-13 16:56:00,964 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:56:00,965 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 03971683bd55396998b5a281a250cba3 in 29ms, sequenceid=19, compaction requested=false 2023-02-13 16:56:00,965 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 03971683bd55396998b5a281a250cba3: 2023-02-13 16:56:00,965 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:00,965 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:00,965 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:00,965 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,965 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:00,965 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:00,971 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,971 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:00,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:00,971 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,971 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:00,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:00,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:00,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:00,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 16:56:00,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@393f8043[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:00,973 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:56:00,973 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:00,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,974 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:00,974 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:00,974 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,974 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:00,974 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:00,974 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:00,974 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:00,974 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:00,975 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:00,974 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:00,976 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:00,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:00,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:00,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:00,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:00,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:00,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,41587,1676307262998': 2023-02-13 16:56:00,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:56:00,979 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 16:56:00,979 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:00,979 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:00,980 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 16:56:00,980 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:00,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,983 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:00,983 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,983 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,984 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,984 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:00,985 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:00,985 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,986 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:00,988 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,988 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:00,988 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:00,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,989 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:00,989 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:00,989 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:00,988 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:00,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:00,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,989 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:00,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:00,989 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:00,989 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,089 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:01,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:01,100 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:01,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:01,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 16:56:01,101 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:01,101 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,101 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 16:56:01,101 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:56:01,102 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,102 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,103 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,103 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,103 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,103 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,103 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,103 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,104 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:01,105 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:01,105 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. started... 2023-02-13 16:56:01,105 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 03971683bd55396998b5a281a250cba3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:01,106 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,108 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,108 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,108 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,108 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,108 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,113 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/26959caa74e64a18aa610bbfd8c26198 2023-02-13 16:56:01,122 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/dced5d29687849958ff78cbc55a2f1cb 2023-02-13 16:56:01,126 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/26959caa74e64a18aa610bbfd8c26198 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198 2023-02-13 16:56:01,129 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:56:01,130 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/dced5d29687849958ff78cbc55a2f1cb as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb 2023-02-13 16:56:01,133 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:56:01,133 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 03971683bd55396998b5a281a250cba3 in 28ms, sequenceid=32, compaction requested=true 2023-02-13 16:56:01,133 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 03971683bd55396998b5a281a250cba3: 2023-02-13 16:56:01,133 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:01,133 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:01,134 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:01,134 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,134 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,134 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,135 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,135 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,135 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,135 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 16:56:01,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2356d550[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:01,137 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:56:01,137 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,138 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:01,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,138 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,138 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:01,138 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,139 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,139 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:01,139 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,139 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,140 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,141 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,141 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,141 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,141 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,141 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,141 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,142 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,41587,1676307262998': 2023-02-13 16:56:01,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:56:01,144 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 16:56:01,145 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:01,145 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:01,145 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,145 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:01,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,147 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,147 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,148 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,149 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,149 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,150 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,156 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:01,156 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,156 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,157 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,156 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,157 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,157 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:01,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,157 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,157 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,157 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,157 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,257 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:01,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:01,268 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:01,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:01,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 16:56:01,269 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:01,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 16:56:01,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:56:01,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,272 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,272 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,272 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,272 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,272 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,272 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:01,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,273 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:01,274 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:01,274 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:01,274 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. started... 2023-02-13 16:56:01,275 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 03971683bd55396998b5a281a250cba3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:01,276 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,277 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,277 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,277 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,277 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,277 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,283 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/7aa7b25ea37942218e070c4533dfc7db 2023-02-13 16:56:01,292 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/9a93c3a53d304e00bc47bf8c96f7ee42 2023-02-13 16:56:01,296 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/7aa7b25ea37942218e070c4533dfc7db as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db 2023-02-13 16:56:01,299 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:56:01,299 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/9a93c3a53d304e00bc47bf8c96f7ee42 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42 2023-02-13 16:56:01,302 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:56:01,303 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 03971683bd55396998b5a281a250cba3 in 28ms, sequenceid=45, compaction requested=true 2023-02-13 16:56:01,303 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 03971683bd55396998b5a281a250cba3: 2023-02-13 16:56:01,303 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:01,303 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:01,303 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:01,303 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,303 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,303 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,305 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,305 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,305 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,305 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 16:56:01,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7862d44f[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:01,307 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:56:01,307 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,309 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:01,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:01,310 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,311 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,311 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,311 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,311 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,311 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,311 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,311 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,311 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,311 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,41587,1676307262998': 2023-02-13 16:56:01,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:56:01,314 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 16:56:01,314 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:01,314 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:01,314 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,314 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:01,316 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,316 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,316 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,316 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,316 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,317 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,317 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,318 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,319 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,319 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,325 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,325 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,325 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,325 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,325 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,325 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:01,325 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:01,325 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,325 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,325 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,326 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,326 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:01,326 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:01,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,326 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,326 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,326 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,326 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,326 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,326 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,327 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,327 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,327 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,426 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:01,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:01,437 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:01,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:01,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 16:56:01,439 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:01,439 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,439 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 16:56:01,439 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:56:01,439 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,439 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,440 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,440 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,440 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,440 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,441 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,441 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,442 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,442 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,442 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,442 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,442 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,442 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,442 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,442 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,442 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,443 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,443 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:01,443 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:01,443 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,443 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,443 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:01,443 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. started... 2023-02-13 16:56:01,443 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,443 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 03971683bd55396998b5a281a250cba3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:01,444 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,445 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,445 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,445 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,445 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,445 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,453 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/6ea076a2ba0042248fbb35e1efbe24d5 2023-02-13 16:56:01,466 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/dd9013c7591548d4b33d5ca2a73a3795 2023-02-13 16:56:01,469 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/6ea076a2ba0042248fbb35e1efbe24d5 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5 2023-02-13 16:56:01,472 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:56:01,473 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/dd9013c7591548d4b33d5ca2a73a3795 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795 2023-02-13 16:56:01,476 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:56:01,477 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 03971683bd55396998b5a281a250cba3 in 34ms, sequenceid=58, compaction requested=true 2023-02-13 16:56:01,477 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 03971683bd55396998b5a281a250cba3: 2023-02-13 16:56:01,477 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,483 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,483 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,483 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,483 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 16:56:01,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15d57b5c[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:01,485 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:56:01,485 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,487 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,487 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:01,487 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:01,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,487 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,487 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,487 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,487 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,487 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:01,487 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:01,489 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,489 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,489 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,489 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,490 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,489 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,490 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,490 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,490 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,490 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,41587,1676307262998': 2023-02-13 16:56:01,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:56:01,495 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 16:56:01,495 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:01,495 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:01,495 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,495 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:01,498 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,498 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,498 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,498 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,499 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,499 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,498 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,499 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,499 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,500 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,502 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,502 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,503 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,505 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,505 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,505 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,505 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,506 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,505 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,506 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:01,506 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:01,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,507 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,507 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,507 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,607 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:01,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:01,619 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:01,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:01,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 16:56:01,621 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:01,621 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,621 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 16:56:01,621 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 16:56:01,621 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,621 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,623 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,623 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,623 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,623 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,623 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,623 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 16:56:01,624 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:01,625 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:01,625 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:01,625 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. started... 2023-02-13 16:56:01,626 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 03971683bd55396998b5a281a250cba3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:01,626 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,628 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,628 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,629 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,629 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,629 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,634 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/99aefd74c9b24cc4b424f0bbe0dc25db 2023-02-13 16:56:01,644 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/ecaaf422a4b44eb6bfb418ef86cf3522 2023-02-13 16:56:01,648 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/99aefd74c9b24cc4b424f0bbe0dc25db as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db 2023-02-13 16:56:01,651 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:56:01,652 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/ecaaf422a4b44eb6bfb418ef86cf3522 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522 2023-02-13 16:56:01,654 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:56:01,655 INFO [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 03971683bd55396998b5a281a250cba3 in 29ms, sequenceid=71, compaction requested=true 2023-02-13 16:56:01,655 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 03971683bd55396998b5a281a250cba3: 2023-02-13 16:56:01,655 DEBUG [rs(jenkins-hbase4.apache.org,41587,1676307262998)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:01,655 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:01,655 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:01,655 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,655 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 16:56:01,655 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 16:56:01,657 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,657 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,657 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,657 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,659 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 16:56:01,659 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50addc3d[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:01,659 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 16:56:01,659 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,660 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,660 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,660 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:01,660 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,660 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,660 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:01,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,660 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,661 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,661 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:01,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,661 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 16:56:01,661 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 16:56:01,661 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:01,662 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,662 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,662 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,663 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,663 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,663 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,663 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,664 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 16:56:01,664 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,664 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 16:56:01,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,41587,1676307262998': 2023-02-13 16:56:01,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 16:56:01,666 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 16:56:01,666 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:01,666 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:01,666 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 16:56:01,666 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:01,668 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,668 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,668 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,668 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,669 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,670 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 16:56:01,671 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,671 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,676 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,677 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 16:56:01,677 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,677 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:01,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:01,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,677 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,678 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:01,678 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,678 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:01,678 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:01,678 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,678 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 16:56:01,778 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:01,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:02,036 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 16:56:02,037 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:02,038 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,039 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,039 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,039 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,039 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:02,040 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:02,040 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:02,288 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,288 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:02,288 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:02,288 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:02,288 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-13 16:56:02,288 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:02,289 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:02,289 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:02,289 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:02,289 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:02,291 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 16:56:02,291 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:02,291 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:02,291 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:02,291 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:02,292 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:02,292 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:02,292 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:02,292 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:02,292 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:02,292 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:02,293 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:02,293 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:02,293 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:02,293 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:02,293 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:02,293 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:02,293 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:02,294 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:02,294 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:02,294 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:02,294 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:02,294 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:02,294 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:02,294 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:02,294 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:02,295 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,295 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,295 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:02,295 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,295 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,296 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:02,296 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:02,296 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:02,296 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:02,296 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,296 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:02,296 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,296 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,296 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,297 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 16:56:02,297 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:02,297 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:02,298 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307392298 2023-02-13 16:56:02,298 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:02,299 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,302 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:02,302 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:02,302 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:02,303 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307422303 2023-02-13 16:56:02,303 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307482303 2023-02-13 16:56:02,303 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:56:02,303 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:02,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:02,305 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:02,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:02,553 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,553 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:02,553 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:02,553 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:02,553 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:02,553 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:02,553 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:02,553 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:02,554 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:02,554 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:02,556 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 16:56:02,556 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:02,557 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:02,557 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:02,557 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:02,557 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:02,558 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:02,558 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:02,558 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:02,558 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:02,559 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:02,559 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:02,559 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:02,559 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:02,560 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:02,560 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:02,560 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:02,560 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:02,560 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:02,561 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:02,561 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:02,561 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:02,561 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:02,561 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:02,561 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:02,561 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:02,561 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,562 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,562 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:02,562 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,562 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,562 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:02,562 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:02,562 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:02,563 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:02,563 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,563 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:02,563 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,563 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,563 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,563 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 16:56:02,564 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:02,564 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:02,565 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307392564 2023-02-13 16:56:02,565 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:02,566 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,568 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:02,568 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:02,568 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:02,568 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307422568 2023-02-13 16:56:02,568 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307482568 2023-02-13 16:56:02,569 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:56:02,569 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:02,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN}] 2023-02-13 16:56:02,570 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:02,570 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:02,571 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:02,819 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,819 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:02,819 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:02,819 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:02,819 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:02,819 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:02,819 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:02,819 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:02,819 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:02,820 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:02,821 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 16:56:02,821 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:02,822 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:02,822 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:02,822 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:02,822 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:02,822 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:02,822 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:02,823 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:02,823 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:02,823 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:02,823 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:02,823 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:02,824 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:02,824 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:02,824 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:02,824 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:02,825 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:02,825 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:02,825 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:02,825 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:02,825 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:02,826 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:02,826 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:02,826 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:02,826 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:02,826 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,827 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,827 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:02,827 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,827 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:02,828 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:02,828 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:02,828 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:02,828 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:02,829 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,829 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:02,829 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,829 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,829 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,829 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 16:56:02,830 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:02,830 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:02,830 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307392830 2023-02-13 16:56:02,831 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:02,832 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:02,834 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:02,834 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:02,834 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:02,834 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307422834 2023-02-13 16:56:02,834 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307482834 2023-02-13 16:56:02,834 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:56:02,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN}] 2023-02-13 16:56:02,835 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:02,836 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN 2023-02-13 16:56:02,837 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=03971683bd55396998b5a281a250cba3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:02,837 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307362837"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307362837"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307362837"}]},"ts":"1676307362837"} 2023-02-13 16:56:02,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998}] 2023-02-13 16:56:02,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:02,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 03971683bd55396998b5a281a250cba3, disabling compactions & flushes 2023-02-13 16:56:02,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:02,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:02,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. after waiting 0 ms 2023-02-13 16:56:02,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:02,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 03971683bd55396998b5a281a250cba3 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 16:56:03,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/b850ab8751294cb994040588982fe8dd 2023-02-13 16:56:03,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/1855e318640d4954993ac28a098b0d0a 2023-02-13 16:56:03,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf1/b850ab8751294cb994040588982fe8dd as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd 2023-02-13 16:56:03,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:56:03,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/.tmp/cf2/1855e318640d4954993ac28a098b0d0a as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a 2023-02-13 16:56:03,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:56:03,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 03971683bd55396998b5a281a250cba3 in 34ms, sequenceid=82, compaction requested=true 2023-02-13 16:56:03,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 16:56:03,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:03,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. 2023-02-13 16:56:03,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 03971683bd55396998b5a281a250cba3: Waiting for close lock at 1676307362990Running coprocessor pre-close hooks at 1676307362990Disabling compacts and flushes for region at 1676307362990Disabling writes for close at 1676307362990Obtaining lock to block concurrent updates at 1676307362990Preparing flush snapshotting stores in 03971683bd55396998b5a281a250cba3 at 1676307362990Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676307362990Flushing stores of testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. at 1676307362991 (+1 ms)Flushing 03971683bd55396998b5a281a250cba3/cf1: creating writer at 1676307362992 (+1 ms)Flushing 03971683bd55396998b5a281a250cba3/cf1: appending metadata at 1676307362994 (+2 ms)Flushing 03971683bd55396998b5a281a250cba3/cf1: closing flushed file at 1676307362994Flushing 03971683bd55396998b5a281a250cba3/cf2: creating writer at 1676307363005 (+11 ms)Flushing 03971683bd55396998b5a281a250cba3/cf2: appending metadata at 1676307363006 (+1 ms)Flushing 03971683bd55396998b5a281a250cba3/cf2: closing flushed file at 1676307363006Flushing 03971683bd55396998b5a281a250cba3/cf1: reopening flushed file at 1676307363018 (+12 ms)Flushing 03971683bd55396998b5a281a250cba3/cf2: reopening flushed file at 1676307363021 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 03971683bd55396998b5a281a250cba3 in 34ms, sequenceid=82, compaction requested=true at 1676307363025 (+4 ms)Writing region close event to WAL at 1676307363026 (+1 ms)Running coprocessor post-close hooks at 1676307363028 (+2 ms)Closed at 1676307363029 (+1 ms) 2023-02-13 16:56:03,031 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,032 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=03971683bd55396998b5a281a250cba3, regionState=CLOSED 2023-02-13 16:56:03,032 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307363032"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307363032"}]},"ts":"1676307363032"} 2023-02-13 16:56:03,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 16:56:03,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 in 195 msec 2023-02-13 16:56:03,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 16:56:03,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN in 199 msec 2023-02-13 16:56:03,037 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:03,037 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:03,038 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:03,085 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,085 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:03,085 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:03,085 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:03,085 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:03,085 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:03,085 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:03,085 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:03,085 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:03,086 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:03,087 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 16:56:03,087 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:56:03,087 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:56:03,087 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:03,088 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:03,088 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:03,088 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:03,088 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:03,088 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:03,088 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:03,088 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:03,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:03,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:03,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:03,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:03,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:03,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:03,089 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:03,090 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:03,090 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:03,090 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:03,090 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:03,090 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:03,090 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:03,090 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:03,091 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:03,091 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:03,091 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:03,091 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:03,092 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:03,092 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:03,092 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:03,092 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:03,092 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:03,092 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:03,092 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:03,093 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:03,093 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,093 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN 2023-02-13 16:56:03,093 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:03,093 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:03,093 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,093 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,093 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,093 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:03,093 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:03,093 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 16:56:03,094 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:03,094 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:03,095 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307393095 2023-02-13 16:56:03,095 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:03,096 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,098 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:03,098 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:03,099 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:03,099 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307423099 2023-02-13 16:56:03,099 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307483099 2023-02-13 16:56:03,099 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:56:03,101 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:03,110 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=03971683bd55396998b5a281a250cba3, threads=14 2023-02-13 16:56:03,110 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,111 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,111 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,110 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,113 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,114 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,114 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,114 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,114 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,114 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,115 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,115 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,115 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,115 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,220 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,220 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,234 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,240 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,244 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,251 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,252 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,253 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,253 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,255 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,255 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,477 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:56:03,648 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,648 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,648 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,648 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 03971683bd55396998b5a281a250cba3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 16:56:03,676 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:03,676 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:03,676 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:03,849 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,849 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:03,849 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:03,849 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:03,850 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:03,850 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:03,850 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:03,850 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:03,850 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:03,850 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:03,851 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 16:56:03,852 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:56:03,852 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:56:03,852 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:03,852 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:03,852 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:03,852 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:03,853 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:03,853 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:03,853 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:03,853 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:03,853 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:03,853 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:03,853 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:03,853 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:03,854 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:03,854 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:03,854 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:03,854 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:03,854 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:03,854 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:03,854 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:03,854 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:03,855 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:03,855 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:03,855 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:03,855 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:03,855 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:03,855 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:03,855 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:03,856 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:03,856 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:03,856 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:03,856 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:03,856 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:03,856 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:03,856 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:03,857 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,857 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN 2023-02-13 16:56:03,857 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:03,857 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:03,857 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,857 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,857 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,857 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:03,857 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:03,857 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 16:56:03,858 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:03,858 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:03,858 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307393858 2023-02-13 16:56:03,860 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:03,860 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:03,862 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:03,862 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:03,862 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:03,862 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307423862 2023-02-13 16:56:03,862 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307483862 2023-02-13 16:56:03,862 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:56:03,871 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=03971683bd55396998b5a281a250cba3, threads=14 2023-02-13 16:56:03,871 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,871 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,871 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,871 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,872 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,872 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,875 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,875 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,875 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,875 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,879 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,879 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,879 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,879 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,937 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,940 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,945 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,948 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,960 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,962 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,964 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,978 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,978 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,978 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,980 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69 for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,980 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,982 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,982 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db for region: 03971683bd55396998b5a281a250cba3 2023-02-13 16:56:03,983 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 03971683bd55396998b5a281a250cba3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 16:56:04,010 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:04,014 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:56:04,016 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:56:04,016 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:04,016 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:04,016 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:04,112 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,113 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:04,113 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:04,113 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:04,113 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:04,113 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:04,113 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:04,113 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:04,113 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:04,114 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:04,115 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 16:56:04,115 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:56:04,115 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:56:04,116 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:04,116 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:04,116 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:04,117 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:04,117 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,117 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:04,117 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,117 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,118 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,118 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:04,118 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,118 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,119 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,119 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:04,119 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:04,119 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,119 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,120 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,120 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:04,120 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,120 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,120 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,121 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:04,121 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,121 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,121 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,121 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,122 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:04,122 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,122 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,122 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,122 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,123 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,123 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:04,123 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,123 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN 2023-02-13 16:56:04,123 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:04,123 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:04,123 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,123 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,123 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,123 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:04,124 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:04,124 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 16:56:04,124 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:04,125 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:04,125 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307394125 2023-02-13 16:56:04,126 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:04,126 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,128 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:04,128 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:04,128 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:04,128 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307424128 2023-02-13 16:56:04,128 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307484128 2023-02-13 16:56:04,129 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:56:04,131 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:04,132 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:04,132 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:04,132 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:04,379 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,379 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:04,379 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:04,379 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:04,379 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:04,379 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:04,379 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:04,379 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:04,379 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:04,380 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:04,381 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 16:56:04,381 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:56:04,381 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:56:04,381 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:04,382 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:04,382 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:04,382 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:04,382 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,382 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:04,382 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,382 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,383 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,383 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:04,383 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,383 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,383 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,383 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:04,383 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:04,383 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,384 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,384 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,384 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:04,384 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,384 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,384 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,384 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:04,384 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,385 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,385 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,385 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,385 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:04,385 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,385 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,385 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,386 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,386 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,386 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:04,386 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,386 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN 2023-02-13 16:56:04,386 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:04,386 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:04,386 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,386 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,386 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,387 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:04,387 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:04,387 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 16:56:04,387 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:04,388 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:04,396 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307394396 2023-02-13 16:56:04,397 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:04,397 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,400 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:04,400 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:04,400 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:04,400 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307424400 2023-02-13 16:56:04,400 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307484400 2023-02-13 16:56:04,400 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 16:56:04,400 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:04,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307364402"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676307364402"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676307364402"}]},"ts":"1676307364402"} 2023-02-13 16:56:04,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307364402"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307364402"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307364402"}]},"ts":"1676307364402"} 2023-02-13 16:56:04,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307364402"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307364402"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307364402"}]},"ts":"1676307364402"} 2023-02-13 16:56:04,404 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:04,404 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:04,404 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:04,650 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,650 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:04,651 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:04,651 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:04,651 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:04,651 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:04,651 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:04,651 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:04,651 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:04,652 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:04,653 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 16:56:04,653 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:56:04,653 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:56:04,654 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:04,654 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:04,654 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:04,654 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:04,655 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,655 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:04,655 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,655 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,656 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,656 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:04,656 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,656 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,656 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,657 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:04,657 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:04,657 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,657 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,657 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,657 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:04,658 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,658 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,658 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,658 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:04,658 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,658 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,659 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,659 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,660 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:04,660 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,660 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,660 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,661 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,661 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,661 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:04,661 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,661 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN 2023-02-13 16:56:04,662 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:04,662 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:04,662 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,662 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,662 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,662 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:04,662 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:04,662 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 16:56:04,663 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:04,663 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:04,664 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307394664 2023-02-13 16:56:04,665 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:04,666 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,670 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:04,671 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:04,671 INFO [Listener at localhost/38547] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3 2023-02-13 16:56:04,671 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=f1a48265333c8d9c41757396d56e06ab, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 16:56:04,671 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=5f59626b2b06658b454ee768885b4581, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 16:56:04,672 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:04,672 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307424672 2023-02-13 16:56:04,672 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307484672 2023-02-13 16:56:04,672 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-13 16:56:04,672 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307364672"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676307364672"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676307364672"}]},"ts":"1676307364672"} 2023-02-13 16:56:04,673 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307364672"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307364672"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307364672"}]},"ts":"1676307364672"} 2023-02-13 16:56:04,673 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307364672"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307364672"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307364672"}]},"ts":"1676307364672"} 2023-02-13 16:56:04,675 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:04,676 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:04,676 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:04,677 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:04,693 ERROR [BP-2104803542-172.31.14.131-1676307258605 heartbeating to localhost/127.0.0.1:33763] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104803542-172.31.14.131-1676307258605 (Datanode Uuid a3585e58-faa4-4a64-84b5-04c6ddf75562) service to localhost/127.0.0.1:33763 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:56:04,922 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,922 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:04,922 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:04,922 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:04,923 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:04,923 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:04,923 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:04,923 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:04,923 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:04,924 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:04,925 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-13 16:56:04,925 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:56:04,925 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:56:04,925 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:04,925 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:04,925 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:04,926 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:04,926 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,926 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:04,926 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,926 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:04,927 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,927 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:04,927 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,927 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:04,927 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,928 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:04,928 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:04,928 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,928 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:04,928 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,928 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:04,928 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,928 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:04,929 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,929 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:04,929 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,929 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:04,929 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,929 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,929 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:04,930 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,930 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:04,930 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,930 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,930 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:04,930 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:04,930 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,930 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN 2023-02-13 16:56:04,930 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:04,931 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:04,931 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,931 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,931 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,931 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:04,931 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:04,931 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 16:56:04,931 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:04,932 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:04,932 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307394932 2023-02-13 16:56:04,932 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:04,934 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,936 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:04,936 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:04,936 INFO [Listener at localhost/38547] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3 2023-02-13 16:56:04,936 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=f1a48265333c8d9c41757396d56e06ab, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 16:56:04,936 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=5f59626b2b06658b454ee768885b4581, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 16:56:04,936 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:04,936 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307424936 2023-02-13 16:56:04,937 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307484937 2023-02-13 16:56:04,937 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:56:04,937 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:04,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, ASSIGN}] 2023-02-13 16:56:04,939 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:04,939 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:04,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:04,939 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:05,187 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,187 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:05,187 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:05,187 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:05,187 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:05,187 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:05,187 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:05,187 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:05,188 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:05,188 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:05,189 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-13 16:56:05,189 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:56:05,189 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:56:05,189 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:05,190 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:05,190 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:05,190 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:05,190 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:05,190 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:05,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:05,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:05,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:05,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:05,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:05,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:05,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:05,191 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:05,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:05,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:05,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:05,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:05,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:05,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:05,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:05,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:05,192 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:05,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:05,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:05,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:05,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:05,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:05,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:05,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:05,193 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:05,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:05,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:05,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:05,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN 2023-02-13 16:56:05,194 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:05,194 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:05,194 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,194 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,194 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,195 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:05,195 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:05,195 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 16:56:05,195 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:05,196 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:05,196 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307395196 2023-02-13 16:56:05,197 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:05,197 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,199 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:05,199 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:05,200 INFO [Listener at localhost/38547] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3 2023-02-13 16:56:05,200 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=f1a48265333c8d9c41757396d56e06ab, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 16:56:05,200 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=5f59626b2b06658b454ee768885b4581, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 16:56:05,200 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:05,200 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307425200 2023-02-13 16:56:05,200 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307485200 2023-02-13 16:56:05,200 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 16:56:05,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, ASSIGN}] 2023-02-13 16:56:05,200 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:05,202 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, ASSIGN 2023-02-13 16:56:05,203 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,41587,1676307262998; forceNewPlan=false, retain=false 2023-02-13 16:56:05,203 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, ASSIGN 2023-02-13 16:56:05,204 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,41587,1676307262998; forceNewPlan=false, retain=false 2023-02-13 16:56:05,353 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 16:56:05,354 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=f1a48265333c8d9c41757396d56e06ab, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:05,354 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307365354"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307365354"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307365354"}]},"ts":"1676307365354"} 2023-02-13 16:56:05,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure f1a48265333c8d9c41757396d56e06ab, server=jenkins-hbase4.apache.org,41587,1676307262998}] 2023-02-13 16:56:05,358 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=5f59626b2b06658b454ee768885b4581, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:05,358 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307365358"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307365358"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307365358"}]},"ts":"1676307365358"} 2023-02-13 16:56:05,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 5f59626b2b06658b454ee768885b4581, server=jenkins-hbase4.apache.org,41587,1676307262998}] 2023-02-13 16:56:05,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:05,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5f59626b2b06658b454ee768885b4581, NAME => 'testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 16:56:05,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:56:05,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:05,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,511 INFO [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,512 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1 2023-02-13 16:56:05,512 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1 2023-02-13 16:56:05,513 INFO [StoreOpener-5f59626b2b06658b454ee768885b4581-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5f59626b2b06658b454ee768885b4581 columnFamilyName cf1 2023-02-13 16:56:05,523 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198-top 2023-02-13 16:56:05,527 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459-top 2023-02-13 16:56:05,529 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5-top 2023-02-13 16:56:05,532 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db-top 2023-02-13 16:56:05,535 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1-top 2023-02-13 16:56:05,538 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db-top 2023-02-13 16:56:05,540 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd-top 2023-02-13 16:56:05,541 INFO [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(310): Store=5f59626b2b06658b454ee768885b4581/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:05,541 INFO [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,541 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2 2023-02-13 16:56:05,541 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2 2023-02-13 16:56:05,542 INFO [StoreOpener-5f59626b2b06658b454ee768885b4581-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5f59626b2b06658b454ee768885b4581 columnFamilyName cf2 2023-02-13 16:56:05,552 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a-top 2023-02-13 16:56:05,555 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e-top 2023-02-13 16:56:05,558 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69-top 2023-02-13 16:56:05,561 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42-top 2023-02-13 16:56:05,564 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb-top 2023-02-13 16:56:05,568 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795-top 2023-02-13 16:56:05,570 DEBUG [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522-top 2023-02-13 16:56:05,571 INFO [StoreOpener-5f59626b2b06658b454ee768885b4581-1] regionserver.HStore(310): Store=5f59626b2b06658b454ee768885b4581/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:05,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:05,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5f59626b2b06658b454ee768885b4581; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60162607, jitterRate=-0.10350729525089264}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:05,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5f59626b2b06658b454ee768885b4581: Running coprocessor pre-open hook at 1676307365511Writing region info on filesystem at 1676307365511Initializing all the Stores at 1676307365511Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307365511Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307365511Cleaning up temporary data from old regions at 1676307365572 (+61 ms)Cleaning up detritus from prior splits at 1676307365573 (+1 ms)Running coprocessor post-open hooks at 1676307365575 (+2 ms)Region opened successfully at 1676307365575 2023-02-13 16:56:05,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581., pid=118, masterSystemTime=1676307365507 2023-02-13 16:56:05,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:05,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:56:05,578 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:05,578 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:05,578 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:05,578 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HStore(1912): 5f59626b2b06658b454ee768885b4581/cf1 is initiating minor compaction (all files) 2023-02-13 16:56:05,578 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:05,578 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HStore(1912): 5f59626b2b06658b454ee768885b4581/cf2 is initiating minor compaction (all files) 2023-02-13 16:56:05,578 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5f59626b2b06658b454ee768885b4581/cf1 in testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:05,578 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5f59626b2b06658b454ee768885b4581/cf2 in testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:05,578 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd-top] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/.tmp, totalSize=34.9 K 2023-02-13 16:56:05,578 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a-top] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/.tmp, totalSize=34.9 K 2023-02-13 16:56:05,579 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307360748 2023-02-13 16:56:05,579 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting 4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307360748 2023-02-13 16:56:05,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:05,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:05,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:05,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f1a48265333c8d9c41757396d56e06ab, NAME => 'testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 16:56:05,579 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307360919 2023-02-13 16:56:05,579 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=5f59626b2b06658b454ee768885b4581, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:05,579 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting 5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307360919 2023-02-13 16:56:05,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:56:05,579 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307365579"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307365579"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307365579"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307365579"}]},"ts":"1676307365579"} 2023-02-13 16:56:05,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:05,579 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307361090 2023-02-13 16:56:05,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:05,579 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307361090 2023-02-13 16:56:05,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:05,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:05,580 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting 9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676307361259 2023-02-13 16:56:05,580 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676307361259 2023-02-13 16:56:05,580 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676307361428 2023-02-13 16:56:05,580 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676307361428 2023-02-13 16:56:05,581 INFO [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:05,581 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676307361608 2023-02-13 16:56:05,581 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676307361608 2023-02-13 16:56:05,581 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting 1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307361779 2023-02-13 16:56:05,581 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307361779 2023-02-13 16:56:05,582 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1 2023-02-13 16:56:05,582 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1 2023-02-13 16:56:05,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-13 16:56:05,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 5f59626b2b06658b454ee768885b4581, server=jenkins-hbase4.apache.org,41587,1676307262998 in 221 msec 2023-02-13 16:56:05,583 INFO [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1a48265333c8d9c41757396d56e06ab columnFamilyName cf1 2023-02-13 16:56:05,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, ASSIGN in 383 msec 2023-02-13 16:56:05,599 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198-bottom 2023-02-13 16:56:05,600 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5f59626b2b06658b454ee768885b4581#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:05,600 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] throttle.PressureAwareThroughputController(145): 5f59626b2b06658b454ee768885b4581#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:05,602 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459-bottom 2023-02-13 16:56:05,609 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5-bottom 2023-02-13 16:56:05,616 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db-bottom 2023-02-13 16:56:05,620 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1-bottom 2023-02-13 16:56:05,620 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/.tmp/cf2/5308b8644d8b49b086db7119e4e70c87 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/5308b8644d8b49b086db7119e4e70c87 2023-02-13 16:56:05,623 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/.tmp/cf1/dd3510fc2fbf491d80559c91e891c923 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/dd3510fc2fbf491d80559c91e891c923 2023-02-13 16:56:05,623 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db-bottom 2023-02-13 16:56:05,625 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5f59626b2b06658b454ee768885b4581/cf2 of 5f59626b2b06658b454ee768885b4581 into 5308b8644d8b49b086db7119e4e70c87(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:05,625 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5f59626b2b06658b454ee768885b4581: 2023-02-13 16:56:05,625 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581., storeName=5f59626b2b06658b454ee768885b4581/cf2, priority=9, startTime=1676307365576; duration=0sec 2023-02-13 16:56:05,625 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:05,627 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5f59626b2b06658b454ee768885b4581/cf1 of 5f59626b2b06658b454ee768885b4581 into dd3510fc2fbf491d80559c91e891c923(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:05,627 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5f59626b2b06658b454ee768885b4581: 2023-02-13 16:56:05,627 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581., storeName=5f59626b2b06658b454ee768885b4581/cf1, priority=9, startTime=1676307365575; duration=0sec 2023-02-13 16:56:05,628 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:05,628 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd-bottom 2023-02-13 16:56:05,628 INFO [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(310): Store=f1a48265333c8d9c41757396d56e06ab/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:05,628 INFO [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:05,628 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2 2023-02-13 16:56:05,629 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2 2023-02-13 16:56:05,629 INFO [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1a48265333c8d9c41757396d56e06ab columnFamilyName cf2 2023-02-13 16:56:05,639 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a-bottom 2023-02-13 16:56:05,641 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e-bottom 2023-02-13 16:56:05,643 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69-bottom 2023-02-13 16:56:05,646 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42-bottom 2023-02-13 16:56:05,648 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb-bottom 2023-02-13 16:56:05,651 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795-bottom 2023-02-13 16:56:05,654 DEBUG [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522-bottom 2023-02-13 16:56:05,654 INFO [StoreOpener-f1a48265333c8d9c41757396d56e06ab-1] regionserver.HStore(310): Store=f1a48265333c8d9c41757396d56e06ab/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:05,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:05,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:05,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:05,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:05,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f1a48265333c8d9c41757396d56e06ab; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66219299, jitterRate=-0.01325555145740509}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:05,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f1a48265333c8d9c41757396d56e06ab: Running coprocessor pre-open hook at 1676307365580Writing region info on filesystem at 1676307365580Initializing all the Stores at 1676307365580Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307365580Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307365580Cleaning up temporary data from old regions at 1676307365656 (+76 ms)Cleaning up detritus from prior splits at 1676307365656Running coprocessor post-open hooks at 1676307365658 (+2 ms)Region opened successfully at 1676307365658 2023-02-13 16:56:05,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab., pid=117, masterSystemTime=1676307365507 2023-02-13 16:56:05,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:05,659 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:05,660 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:05,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:05,660 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:05,660 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:05,660 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HStore(1912): f1a48265333c8d9c41757396d56e06ab/cf1 is initiating minor compaction (all files) 2023-02-13 16:56:05,660 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HStore(1912): f1a48265333c8d9c41757396d56e06ab/cf2 is initiating minor compaction (all files) 2023-02-13 16:56:05,660 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f1a48265333c8d9c41757396d56e06ab/cf1 in testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:05,660 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f1a48265333c8d9c41757396d56e06ab/cf2 in testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:05,660 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/.tmp, totalSize=34.9 K 2023-02-13 16:56:05,660 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/.tmp, totalSize=34.9 K 2023-02-13 16:56:05,661 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307360748 2023-02-13 16:56:05,661 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting 4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307360748 2023-02-13 16:56:05,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:05,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:05,661 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307360919 2023-02-13 16:56:05,661 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting 5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307360919 2023-02-13 16:56:05,661 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=f1a48265333c8d9c41757396d56e06ab, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:05,662 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307365661"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307365661"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307365661"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307365661"}]},"ts":"1676307365661"} 2023-02-13 16:56:05,662 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307361090 2023-02-13 16:56:05,662 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307361090 2023-02-13 16:56:05,662 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting 9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307361259 2023-02-13 16:56:05,662 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307361259 2023-02-13 16:56:05,662 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307361428 2023-02-13 16:56:05,663 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307361428 2023-02-13 16:56:05,663 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307361608 2023-02-13 16:56:05,663 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting 99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307361608 2023-02-13 16:56:05,663 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] compactions.Compactor(207): Compacting 1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307361779 2023-02-13 16:56:05,663 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] compactions.Compactor(207): Compacting b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307361779 2023-02-13 16:56:05,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-13 16:56:05,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure f1a48265333c8d9c41757396d56e06ab, server=jenkins-hbase4.apache.org,41587,1676307262998 in 307 msec 2023-02-13 16:56:05,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-13 16:56:05,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, ASSIGN in 465 msec 2023-02-13 16:56:05,666 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 16:56:05,666 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:05,666 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 16:56:05,673 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] throttle.PressureAwareThroughputController(145): f1a48265333c8d9c41757396d56e06ab#cf2#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:05,673 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] throttle.PressureAwareThroughputController(145): f1a48265333c8d9c41757396d56e06ab#cf1#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:05,690 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/.tmp/cf2/6a3d6d87b4ac4ec5a8cdfb8eaa08f9b4 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/6a3d6d87b4ac4ec5a8cdfb8eaa08f9b4 2023-02-13 16:56:05,691 ERROR [BP-2104803542-172.31.14.131-1676307258605 heartbeating to localhost/127.0.0.1:33763] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104803542-172.31.14.131-1676307258605 (Datanode Uuid 1fadd7f6-283b-4c52-81b3-6a5231ac6c2e) service to localhost/127.0.0.1:33763 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:56:05,691 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/.tmp/cf1/2f8a900551744aab87a88c3c3cb8e283 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/2f8a900551744aab87a88c3c3cb8e283 2023-02-13 16:56:05,695 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f1a48265333c8d9c41757396d56e06ab/cf2 of f1a48265333c8d9c41757396d56e06ab into 6a3d6d87b4ac4ec5a8cdfb8eaa08f9b4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:05,695 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f1a48265333c8d9c41757396d56e06ab: 2023-02-13 16:56:05,695 INFO [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab., storeName=f1a48265333c8d9c41757396d56e06ab/cf2, priority=9, startTime=1676307365659; duration=0sec 2023-02-13 16:56:05,695 DEBUG [RS:0;jenkins-hbase4:41587-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:05,696 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f1a48265333c8d9c41757396d56e06ab/cf1 of f1a48265333c8d9c41757396d56e06ab into 2f8a900551744aab87a88c3c3cb8e283(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:05,696 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f1a48265333c8d9c41757396d56e06ab: 2023-02-13 16:56:05,696 INFO [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab., storeName=f1a48265333c8d9c41757396d56e06ab/cf1, priority=9, startTime=1676307365659; duration=0sec 2023-02-13 16:56:05,696 DEBUG [RS:0;jenkins-hbase4:41587-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:05,700 INFO [Listener at localhost/38547] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,700 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 16:56:05,701 INFO [Listener at localhost/38547] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:05,701 INFO [Listener at localhost/38547] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:05,701 ERROR [Listener at localhost/38547] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 16:56:05,701 INFO [Listener at localhost/38547] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 16:56:05,701 INFO [Listener at localhost/38547] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 16:56:05,701 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 16:56:05,701 INFO [Listener at localhost/38547] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:56:05,702 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 16:56:05,703 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-13 16:56:05,703 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 16:56:05,704 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 16:56:05,704 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-02-13 16:56:05,704 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-02-13 16:56:05,704 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-02-13 16:56:05,704 DEBUG [Listener at localhost/38547] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-02-13 16:56:05,704 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 16:56:05,705 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 16:56:05,705 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 16:56:05,705 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:56:05,705 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:05,705 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=13c13f79b91b6c01ec9b3edbb20ead87, daughterA=fea250aff2133b2f109bfd4e8765243e, daughterB=8b86be70872f3798a9e95e2f1e613a73 2023-02-13 16:56:05,706 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:05,706 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 16:56:05,706 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:05,706 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7c340e1b244f9866f9e7640b1e7f78ea, daughterA=11ec0bb6bbbf4d006fe9a7fb93bd9046, daughterB=4cc157038f0cfa34e35dda26617b6793 2023-02-13 16:56:05,706 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:05,707 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 16:56:05,707 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:05,707 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1ad9073aa1705b085f9e78eda1933547, daughterA=2b1539d8e8a20fc139b22fb9633eb840, daughterB=c7e558972556a4755a23d3e955b540d9 2023-02-13 16:56:05,707 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1ad9073aa1705b085f9e78eda1933547, ASSIGN 2023-02-13 16:56:05,708 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:05,708 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 16:56:05,708 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:05,708 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=9a580bc12d3425731fe759637781ddc1, daughterA=8261b510c81f764ac5b4f1f2b9db29aa, daughterB=ac409e0d07945beddac77394f7974c3c 2023-02-13 16:56:05,708 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:05,708 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 16:56:05,709 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:05,709 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7ad67c2797888267addece4f08ec7def, daughterA=9a11199978f3b74def14fa827748dbd2, daughterB=8a9ff403ae971d224cfffc5e5e7ac123 2023-02-13 16:56:05,709 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:05,709 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 16:56:05,709 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:05,710 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:05,710 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRollbackForSplitTableRegionWithReplica, region=6b6f20b9cb1f3f3857c81c514fc6d98a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=60a6f7bdba888b67b70e7783b1b80897, daughterA=100a97c4ab72df0a94bd6343755f8247, daughterB=d072aae7ac4f571f83ce07078e230e2f 2023-02-13 16:56:05,710 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:05,710 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 16:56:05,710 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:05,711 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:05,711 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 16:56:05,711 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:05,712 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,713 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN 2023-02-13 16:56:05,713 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:05,713 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, ASSIGN 2023-02-13 16:56:05,713 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, ASSIGN 2023-02-13 16:56:05,713 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure f1a48265333c8d9c41757396d56e06ab, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:05,713 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 5f59626b2b06658b454ee768885b4581, server=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:05,713 DEBUG [Listener at localhost/38547] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 held the lock before restarting, call acquireLock to restore it. 2023-02-13 16:56:05,713 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,713 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,714 INFO [Listener at localhost/38547] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,714 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:05,714 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 03971683bd55396998b5a281a250cba3, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:05,714 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:05,714 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:05,714 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure f1a48265333c8d9c41757396d56e06ab, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:05,714 DEBUG [Listener at localhost/38547] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 5f59626b2b06658b454ee768885b4581, server=jenkins-hbase4.apache.org,41587,1676307262998 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 16:56:05,714 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-13 16:56:05,715 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 16:56:05,716 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076, table=hbase:meta, region=1588230740 2023-02-13 16:56:05,718 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307395718 2023-02-13 16:56:05,720 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:05,720 DEBUG [Listener at localhost/38547] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:56:05,723 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=49a9767f24746f442cea05a1662ed767, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38051,1676307263041, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041, openSeqNum=2 2023-02-13 16:56:05,724 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=03971683bd55396998b5a281a250cba3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=2 2023-02-13 16:56:05,724 INFO [Listener at localhost/38547] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,41587,1676307262998, table=testRecoveryAndDoubleExecution, region=03971683bd55396998b5a281a250cba3 2023-02-13 16:56:05,724 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=f1a48265333c8d9c41757396d56e06ab, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=86 2023-02-13 16:56:05,724 DEBUG [Listener at localhost/38547] assignment.RegionStateStore(147): Load hbase:meta entry region=5f59626b2b06658b454ee768885b4581, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41587,1676307262998, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998, openSeqNum=86 2023-02-13 16:56:05,724 INFO [Listener at localhost/38547] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 16:56:05,724 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307425724 2023-02-13 16:56:05,725 INFO [Listener at localhost/38547] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307485725 2023-02-13 16:56:05,725 INFO [Listener at localhost/38547] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 16:56:05,725 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 16:56:05,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=03971683bd55396998b5a281a250cba3, daughterA=f1a48265333c8d9c41757396d56e06ab, daughterB=5f59626b2b06658b454ee768885b4581 in 3.6880 sec 2023-02-13 16:56:05,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41587] ipc.CallRunner(144): callId: 896 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:32860 deadline: 1676307425976, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3. is not online on jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:06,087 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 16:56:06,094 INFO [Listener at localhost/38547] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 16:56:06,095 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 16:56:06,103 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:56:06,104 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-13 16:56:06,105 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-13 16:56:06,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:06,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 16:56:06,108 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307366108"}]},"ts":"1676307366108"} 2023-02-13 16:56:06,109 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 16:56:06,111 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-13 16:56:06,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, UNASSIGN}] 2023-02-13 16:56:06,112 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, UNASSIGN 2023-02-13 16:56:06,112 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=5f59626b2b06658b454ee768885b4581, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:06,113 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307366112"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307366112"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307366112"}]},"ts":"1676307366112"} 2023-02-13 16:56:06,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 5f59626b2b06658b454ee768885b4581, server=jenkins-hbase4.apache.org,41587,1676307262998}] 2023-02-13 16:56:06,115 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, UNASSIGN 2023-02-13 16:56:06,116 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=f1a48265333c8d9c41757396d56e06ab, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:06,116 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307366116"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307366116"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307366116"}]},"ts":"1676307366116"} 2023-02-13 16:56:06,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure f1a48265333c8d9c41757396d56e06ab, server=jenkins-hbase4.apache.org,41587,1676307262998}] 2023-02-13 16:56:06,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 16:56:06,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:06,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5f59626b2b06658b454ee768885b4581, disabling compactions & flushes 2023-02-13 16:56:06,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:06,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:06,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. after waiting 0 ms 2023-02-13 16:56:06,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:06,266 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd-top] to archive 2023-02-13 16:56:06,267 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:06,268 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,269 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,270 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,271 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,272 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,273 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,274 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,275 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a-top] to archive 2023-02-13 16:56:06,275 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:06,277 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,278 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,278 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,279 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,280 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,281 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,282 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:56:06,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:06,286 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581. 2023-02-13 16:56:06,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5f59626b2b06658b454ee768885b4581: Waiting for close lock at 1676307366265Running coprocessor pre-close hooks at 1676307366265Disabling compacts and flushes for region at 1676307366265Disabling writes for close at 1676307366265Writing region close event to WAL at 1676307366283 (+18 ms)Running coprocessor post-close hooks at 1676307366286 (+3 ms)Closed at 1676307366286 2023-02-13 16:56:06,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:06,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:06,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f1a48265333c8d9c41757396d56e06ab, disabling compactions & flushes 2023-02-13 16:56:06,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:06,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:06,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. after waiting 0 ms 2023-02-13 16:56:06,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:06,288 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=5f59626b2b06658b454ee768885b4581, regionState=CLOSED 2023-02-13 16:56:06,288 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307366288"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307366288"}]},"ts":"1676307366288"} 2023-02-13 16:56:06,289 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd-bottom] to archive 2023-02-13 16:56:06,289 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:06,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-13 16:56:06,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 5f59626b2b06658b454ee768885b4581, server=jenkins-hbase4.apache.org,41587,1676307262998 in 176 msec 2023-02-13 16:56:06,291 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5f59626b2b06658b454ee768885b4581, UNASSIGN in 180 msec 2023-02-13 16:56:06,292 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,293 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,294 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,295 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,295 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,296 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,298 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a-bottom] to archive 2023-02-13 16:56:06,298 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:06,299 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,300 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,301 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,302 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,303 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,304 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,305 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:56:06,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:06,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab. 2023-02-13 16:56:06,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f1a48265333c8d9c41757396d56e06ab: Waiting for close lock at 1676307366288Running coprocessor pre-close hooks at 1676307366288Disabling compacts and flushes for region at 1676307366288Disabling writes for close at 1676307366288Writing region close event to WAL at 1676307366305 (+17 ms)Running coprocessor post-close hooks at 1676307366308 (+3 ms)Closed at 1676307366308 2023-02-13 16:56:06,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:06,310 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=f1a48265333c8d9c41757396d56e06ab, regionState=CLOSED 2023-02-13 16:56:06,310 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307366310"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307366310"}]},"ts":"1676307366310"} 2023-02-13 16:56:06,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-13 16:56:06,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure f1a48265333c8d9c41757396d56e06ab, server=jenkins-hbase4.apache.org,41587,1676307262998 in 194 msec 2023-02-13 16:56:06,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-13 16:56:06,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f1a48265333c8d9c41757396d56e06ab, UNASSIGN in 202 msec 2023-02-13 16:56:06,315 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307366315"}]},"ts":"1676307366315"} 2023-02-13 16:56:06,316 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 16:56:06,317 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-13 16:56:06,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 212 msec 2023-02-13 16:56:06,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 16:56:06,410 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-13 16:56:06,411 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-13 16:56:06,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:06,413 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:06,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 16:56:06,414 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:06,417 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,418 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:06,418 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:06,420 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/recovered.edits] 2023-02-13 16:56:06,421 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/recovered.edits] 2023-02-13 16:56:06,421 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/recovered.edits] 2023-02-13 16:56:06,424 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/2f8a900551744aab87a88c3c3cb8e283 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf1/2f8a900551744aab87a88c3c3cb8e283 2023-02-13 16:56:06,425 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/.regioninfo to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/.regioninfo 2023-02-13 16:56:06,425 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/26959caa74e64a18aa610bbfd8c26198 2023-02-13 16:56:06,426 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/64fe6bc523d7443495e81ec2e08af459 2023-02-13 16:56:06,426 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/6a3d6d87b4ac4ec5a8cdfb8eaa08f9b4 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/cf2/6a3d6d87b4ac4ec5a8cdfb8eaa08f9b4 2023-02-13 16:56:06,428 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/6ea076a2ba0042248fbb35e1efbe24d5 2023-02-13 16:56:06,428 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,430 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7aa7b25ea37942218e070c4533dfc7db 2023-02-13 16:56:06,431 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/.regioninfo to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/.regioninfo 2023-02-13 16:56:06,431 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,432 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1 2023-02-13 16:56:06,432 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,433 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/99aefd74c9b24cc4b424f0bbe0dc25db 2023-02-13 16:56:06,433 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,434 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/26959caa74e64a18aa610bbfd8c26198.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,434 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf1/b850ab8751294cb994040588982fe8dd 2023-02-13 16:56:06,435 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,436 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/64fe6bc523d7443495e81ec2e08af459.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,436 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,437 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/6ea076a2ba0042248fbb35e1efbe24d5.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,437 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,437 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/1855e318640d4954993ac28a098b0d0a 2023-02-13 16:56:06,438 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/7aa7b25ea37942218e070c4533dfc7db.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,438 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/4c0969c0db6b4e8fb10dab98eb864d2e 2023-02-13 16:56:06,440 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/5cfd5794d1f848b2ba61abf4dd522a69 2023-02-13 16:56:06,440 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/7e1b73a6a9ac4e1ab1bbf3bbf198f5f1.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,440 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,441 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/99aefd74c9b24cc4b424f0bbe0dc25db.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,441 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/9a93c3a53d304e00bc47bf8c96f7ee42 2023-02-13 16:56:06,441 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,442 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf1/b850ab8751294cb994040588982fe8dd.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,442 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dced5d29687849958ff78cbc55a2f1cb 2023-02-13 16:56:06,442 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,443 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/dd9013c7591548d4b33d5ca2a73a3795 2023-02-13 16:56:06,444 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,445 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/cf2/ecaaf422a4b44eb6bfb418ef86cf3522 2023-02-13 16:56:06,445 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/1855e318640d4954993ac28a098b0d0a.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,445 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,446 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/4c0969c0db6b4e8fb10dab98eb864d2e.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,446 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,447 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/recovered.edits/85.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3/recovered.edits/85.seqid 2023-02-13 16:56:06,447 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/5cfd5794d1f848b2ba61abf4dd522a69.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,447 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/5f59626b2b06658b454ee768885b4581/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,448 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,449 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/9a93c3a53d304e00bc47bf8c96f7ee42.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,450 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/dd3510fc2fbf491d80559c91e891c923 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf1/dd3510fc2fbf491d80559c91e891c923 2023-02-13 16:56:06,450 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/dced5d29687849958ff78cbc55a2f1cb.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,451 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/dd9013c7591548d4b33d5ca2a73a3795.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,452 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/5308b8644d8b49b086db7119e4e70c87 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/cf2/5308b8644d8b49b086db7119e4e70c87 2023-02-13 16:56:06,452 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/f1a48265333c8d9c41757396d56e06ab/cf2/ecaaf422a4b44eb6bfb418ef86cf3522.03971683bd55396998b5a281a250cba3 2023-02-13 16:56:06,454 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581/recovered.edits/90.seqid 2023-02-13 16:56:06,455 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab/recovered.edits/90.seqid 2023-02-13 16:56:06,455 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/5f59626b2b06658b454ee768885b4581 2023-02-13 16:56:06,455 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testRecoveryAndDoubleExecution/f1a48265333c8d9c41757396d56e06ab 2023-02-13 16:56:06,455 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 16:56:06,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:06,459 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-13 16:56:06,460 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-13 16:56:06,461 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:06,461 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-13 16:56:06,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307366461"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:06,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307366461"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:06,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307366461"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:06,463 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 16:56:06,463 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 03971683bd55396998b5a281a250cba3, NAME => 'testRecoveryAndDoubleExecution,,1676307360240.03971683bd55396998b5a281a250cba3.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => f1a48265333c8d9c41757396d56e06ab, NAME => 'testRecoveryAndDoubleExecution,,1676307362037.f1a48265333c8d9c41757396d56e06ab.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 5f59626b2b06658b454ee768885b4581, NAME => 'testRecoveryAndDoubleExecution,41,1676307362037.5f59626b2b06658b454ee768885b4581.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 16:56:06,463 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-13 16:56:06,463 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307366463"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:06,464 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-13 16:56:06,465 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 16:56:06,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 54 msec 2023-02-13 16:56:06,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 16:56:06,515 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-13 16:56:06,529 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=375 (was 364) - Thread LEAK? -, OpenFileDescriptor=832 (was 735) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=108 (was 117), ProcessCount=168 (was 168), AvailableMemoryMB=1040 (was 1099) 2023-02-13 16:56:06,543 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375, OpenFileDescriptor=832, MaxFileDescriptor=60000, SystemLoadAverage=108, ProcessCount=168, AvailableMemoryMB=1040 2023-02-13 16:56:06,543 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:56:06,546 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:56:06,547 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:56:06,548 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 16:56:06,549 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:56:06,549 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:56:06,551 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,551 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44 empty. 2023-02-13 16:56:06,552 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,552 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 16:56:06,560 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 16:56:06,561 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 13acdfb7d6e238f0eec569bef8b8af44, NAME => 'testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:56:06,567 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:06,567 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 13acdfb7d6e238f0eec569bef8b8af44, disabling compactions & flushes 2023-02-13 16:56:06,567 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:06,567 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:06,567 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. after waiting 0 ms 2023-02-13 16:56:06,567 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:06,567 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:06,567 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 13acdfb7d6e238f0eec569bef8b8af44: Waiting for close lock at 1676307366567Disabling compacts and flushes for region at 1676307366567Disabling writes for close at 1676307366567Writing region close event to WAL at 1676307366567Closed at 1676307366567 2023-02-13 16:56:06,569 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:56:06,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307366569"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307366569"}]},"ts":"1676307366569"} 2023-02-13 16:56:06,570 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:56:06,571 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:56:06,571 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307366571"}]},"ts":"1676307366571"} 2023-02-13 16:56:06,572 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-13 16:56:06,574 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:56:06,575 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:56:06,575 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:56:06,575 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:56:06,575 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:56:06,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=13acdfb7d6e238f0eec569bef8b8af44, ASSIGN}] 2023-02-13 16:56:06,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=13acdfb7d6e238f0eec569bef8b8af44, ASSIGN 2023-02-13 16:56:06,577 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=13acdfb7d6e238f0eec569bef8b8af44, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:56:06,727 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:56:06,728 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=13acdfb7d6e238f0eec569bef8b8af44, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:06,728 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307366728"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307366728"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307366728"}]},"ts":"1676307366728"} 2023-02-13 16:56:06,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 13acdfb7d6e238f0eec569bef8b8af44, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:56:06,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:06,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 13acdfb7d6e238f0eec569bef8b8af44, NAME => 'testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:56:06,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:56:06,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:06,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,885 INFO [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,886 DEBUG [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1 2023-02-13 16:56:06,886 DEBUG [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1 2023-02-13 16:56:06,886 INFO [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 13acdfb7d6e238f0eec569bef8b8af44 columnFamilyName cf1 2023-02-13 16:56:06,887 INFO [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] regionserver.HStore(310): Store=13acdfb7d6e238f0eec569bef8b8af44/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:06,887 INFO [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,888 DEBUG [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2 2023-02-13 16:56:06,888 DEBUG [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2 2023-02-13 16:56:06,888 INFO [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 13acdfb7d6e238f0eec569bef8b8af44 columnFamilyName cf2 2023-02-13 16:56:06,889 INFO [StoreOpener-13acdfb7d6e238f0eec569bef8b8af44-1] regionserver.HStore(310): Store=13acdfb7d6e238f0eec569bef8b8af44/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:06,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:06,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:06,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:56:06,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 13acdfb7d6e238f0eec569bef8b8af44; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73527405, jitterRate=0.09564371407032013}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:06,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 13acdfb7d6e238f0eec569bef8b8af44: Running coprocessor pre-open hook at 1676307366884Writing region info on filesystem at 1676307366884Initializing all the Stores at 1676307366885 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307366885Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307366885Cleaning up temporary data from old regions at 1676307366890 (+5 ms)Cleaning up detritus from prior splits at 1676307366890Running coprocessor post-open hooks at 1676307366893 (+3 ms)Region opened successfully at 1676307366893 2023-02-13 16:56:06,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44., pid=127, masterSystemTime=1676307366881 2023-02-13 16:56:06,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:06,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:06,896 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=13acdfb7d6e238f0eec569bef8b8af44, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:06,896 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307366896"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307366896"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307366896"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307366896"}]},"ts":"1676307366896"} 2023-02-13 16:56:06,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-13 16:56:06,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 13acdfb7d6e238f0eec569bef8b8af44, server=jenkins-hbase4.apache.org,34717,1676307263076 in 167 msec 2023-02-13 16:56:06,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-13 16:56:06,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=13acdfb7d6e238f0eec569bef8b8af44, ASSIGN in 323 msec 2023-02-13 16:56:06,900 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:56:06,900 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307366900"}]},"ts":"1676307366900"} 2023-02-13 16:56:06,901 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-13 16:56:06,903 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:56:06,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 356 msec 2023-02-13 16:56:07,055 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:07,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,056 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,056 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,056 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 16:56:07,056 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 16:56:07,057 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,057 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,059 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,059 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,059 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,059 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,059 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,059 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:07,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,060 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,061 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,061 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,061 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,061 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. started... 2023-02-13 16:56:07,062 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13acdfb7d6e238f0eec569bef8b8af44 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 16:56:07,063 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,063 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,063 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,063 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,063 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,063 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,071 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/c945d8626f904fc485dbb9d4f38a6d36 2023-02-13 16:56:07,081 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/d2d8291a943c4081976c19d4810e4f65 2023-02-13 16:56:07,084 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/c945d8626f904fc485dbb9d4f38a6d36 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/c945d8626f904fc485dbb9d4f38a6d36 2023-02-13 16:56:07,087 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/c945d8626f904fc485dbb9d4f38a6d36, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:56:07,088 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/d2d8291a943c4081976c19d4810e4f65 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/d2d8291a943c4081976c19d4810e4f65 2023-02-13 16:56:07,090 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/d2d8291a943c4081976c19d4810e4f65, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:56:07,091 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 13acdfb7d6e238f0eec569bef8b8af44 in 29ms, sequenceid=6, compaction requested=false 2023-02-13 16:56:07,091 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,092 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13acdfb7d6e238f0eec569bef8b8af44: 2023-02-13 16:56:07,092 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,092 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:07,092 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:07,092 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,092 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,092 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,093 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,093 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,093 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,094 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 16:56:07,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45447f2d[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:07,095 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 16:56:07,095 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,096 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:07,097 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,097 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,097 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:07,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,096 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,097 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:07,097 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,097 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,097 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:07,099 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,099 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,099 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,099 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,100 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,100 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,100 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,100 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:56:07,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:56:07,103 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 16:56:07,103 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:07,103 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:07,103 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,103 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:07,104 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,104 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,104 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,105 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,105 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,106 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,107 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,107 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,112 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,112 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,112 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,112 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,112 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,112 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,112 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,113 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,113 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,112 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,113 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:07,112 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,112 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,113 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,113 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:07,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,113 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,113 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,113 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:07,113 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,114 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,114 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,114 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,214 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:07,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:07,225 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:07,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:07,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,226 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,226 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,226 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 16:56:07,226 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 16:56:07,227 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,227 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,228 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,228 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,228 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,228 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,228 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,228 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:07,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:07,230 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,230 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. started... 2023-02-13 16:56:07,230 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13acdfb7d6e238f0eec569bef8b8af44 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:07,234 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,234 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,234 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,234 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,235 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,235 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,239 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/d9ffd4875ad34a8da2b4a331a59d7a02 2023-02-13 16:56:07,248 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/c50cf8f9428a4434bcacf9a410328542 2023-02-13 16:56:07,252 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/d9ffd4875ad34a8da2b4a331a59d7a02 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/d9ffd4875ad34a8da2b4a331a59d7a02 2023-02-13 16:56:07,255 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/d9ffd4875ad34a8da2b4a331a59d7a02, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:56:07,256 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/c50cf8f9428a4434bcacf9a410328542 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/c50cf8f9428a4434bcacf9a410328542 2023-02-13 16:56:07,259 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/c50cf8f9428a4434bcacf9a410328542, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:56:07,259 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13acdfb7d6e238f0eec569bef8b8af44 in 29ms, sequenceid=19, compaction requested=false 2023-02-13 16:56:07,260 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13acdfb7d6e238f0eec569bef8b8af44: 2023-02-13 16:56:07,260 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:07,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:07,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,260 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,261 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,261 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,262 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,262 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 16:56:07,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bea1b66[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:07,263 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 16:56:07,263 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,264 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,264 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,264 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,265 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,265 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:07,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:07,267 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,267 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,267 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,267 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,267 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,267 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,268 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,268 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,268 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,268 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:56:07,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:56:07,271 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 16:56:07,271 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:07,271 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:07,271 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,271 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:07,272 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,272 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,272 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,272 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,272 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,273 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,273 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,273 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,273 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,274 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,275 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,275 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,275 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,278 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,278 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,278 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:07,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,278 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,279 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,279 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:07,279 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,279 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:07,279 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,279 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,279 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,279 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,279 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,379 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:07,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:07,390 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:07,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:07,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,391 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,391 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,391 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 16:56:07,391 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 16:56:07,392 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,392 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,394 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,394 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,394 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,394 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,394 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,395 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,395 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,396 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,396 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,396 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,396 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,396 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. started... 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,396 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13acdfb7d6e238f0eec569bef8b8af44 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,397 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,397 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,397 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,409 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/1d59cc3113164feab2f0bfff49401662 2023-02-13 16:56:07,418 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/045baaca582b44d8ac74ddd767904a55 2023-02-13 16:56:07,421 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/1d59cc3113164feab2f0bfff49401662 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/1d59cc3113164feab2f0bfff49401662 2023-02-13 16:56:07,424 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/1d59cc3113164feab2f0bfff49401662, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:56:07,425 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/045baaca582b44d8ac74ddd767904a55 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/045baaca582b44d8ac74ddd767904a55 2023-02-13 16:56:07,428 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/045baaca582b44d8ac74ddd767904a55, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:56:07,429 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13acdfb7d6e238f0eec569bef8b8af44 in 33ms, sequenceid=32, compaction requested=true 2023-02-13 16:56:07,429 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13acdfb7d6e238f0eec569bef8b8af44: 2023-02-13 16:56:07,429 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,429 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:07,429 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:07,429 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,429 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,429 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,431 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,431 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,431 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,431 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 16:56:07,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@155b8adc[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:07,433 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 16:56:07,433 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,435 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,435 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:07,435 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,435 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,435 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:07,435 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,435 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,436 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,436 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:07,436 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:07,443 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,443 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,443 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,443 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,443 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,443 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,443 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,443 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,443 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,443 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:56:07,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:56:07,446 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 16:56:07,446 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:07,446 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:07,446 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,446 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:07,447 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,447 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,447 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,447 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,447 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,447 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,448 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,448 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,449 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,449 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,450 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,451 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,451 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,454 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,454 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,454 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,454 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,454 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,455 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,455 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:07,455 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,454 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,454 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,454 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,455 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:07,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,455 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:07,455 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,455 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,456 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,456 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,456 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,456 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,456 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,456 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,456 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,555 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:07,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:07,566 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:07,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:07,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,567 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,567 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,567 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 16:56:07,567 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 16:56:07,567 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,567 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,568 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,568 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,568 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,568 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,569 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,569 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,570 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,570 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,570 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,570 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,570 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:07,571 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,571 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. started... 2023-02-13 16:56:07,571 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13acdfb7d6e238f0eec569bef8b8af44 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,579 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/982408db99074b45b40af2e2f2c24dfa 2023-02-13 16:56:07,588 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/59834d280d9d4e718b6ff7d9263dc9c7 2023-02-13 16:56:07,591 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/982408db99074b45b40af2e2f2c24dfa as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/982408db99074b45b40af2e2f2c24dfa 2023-02-13 16:56:07,594 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/982408db99074b45b40af2e2f2c24dfa, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:56:07,594 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/59834d280d9d4e718b6ff7d9263dc9c7 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/59834d280d9d4e718b6ff7d9263dc9c7 2023-02-13 16:56:07,597 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/59834d280d9d4e718b6ff7d9263dc9c7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:56:07,598 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13acdfb7d6e238f0eec569bef8b8af44 in 27ms, sequenceid=45, compaction requested=true 2023-02-13 16:56:07,598 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13acdfb7d6e238f0eec569bef8b8af44: 2023-02-13 16:56:07,598 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,598 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:07,598 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:07,598 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,598 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,598 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,599 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,599 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,600 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,600 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 16:56:07,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b29e07e[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:07,601 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 16:56:07,601 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,603 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,603 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,603 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,603 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,604 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,604 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:07,603 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,604 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:07,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,604 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,604 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,604 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,604 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:07,604 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,604 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:07,605 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,605 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,605 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,606 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,606 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,606 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,606 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,606 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,606 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,606 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:56:07,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:56:07,609 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 16:56:07,609 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:07,609 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:07,609 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,609 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:07,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,610 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,611 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,611 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,612 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,613 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,613 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,614 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,624 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,624 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,624 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,624 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,624 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,625 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,625 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:07,625 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,624 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,624 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,625 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,625 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,625 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:07,625 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:07,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,625 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,625 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,625 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,625 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,626 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,626 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,626 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,626 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,725 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:07,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:07,736 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:07,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:07,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,737 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,737 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,737 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 16:56:07,737 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 16:56:07,738 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,738 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,739 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,739 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,739 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,739 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,739 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,739 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:07,741 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:07,741 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,742 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. started... 2023-02-13 16:56:07,742 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13acdfb7d6e238f0eec569bef8b8af44 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:07,742 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,749 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/8635cc636eae4d6681e3dbffd4306864 2023-02-13 16:56:07,760 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/8d6b0bd20e724c1f8235f83db035462d 2023-02-13 16:56:07,763 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/8635cc636eae4d6681e3dbffd4306864 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/8635cc636eae4d6681e3dbffd4306864 2023-02-13 16:56:07,767 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/8635cc636eae4d6681e3dbffd4306864, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:56:07,767 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/8d6b0bd20e724c1f8235f83db035462d as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/8d6b0bd20e724c1f8235f83db035462d 2023-02-13 16:56:07,770 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/8d6b0bd20e724c1f8235f83db035462d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:56:07,771 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13acdfb7d6e238f0eec569bef8b8af44 in 29ms, sequenceid=58, compaction requested=true 2023-02-13 16:56:07,771 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13acdfb7d6e238f0eec569bef8b8af44: 2023-02-13 16:56:07,771 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,771 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:07,771 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:07,771 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,772 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,772 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,773 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,773 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,774 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,774 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 16:56:07,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4df25f21[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:07,776 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 16:56:07,776 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,778 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,778 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:07,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:07,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,778 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:07,779 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,780 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,780 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,780 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,780 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,780 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,780 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,782 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,782 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,782 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:56:07,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:56:07,785 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 16:56:07,785 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:07,785 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:07,785 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,785 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:07,786 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,786 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,786 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,786 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,787 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,786 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,787 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,787 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,787 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,788 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,789 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,789 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,790 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,792 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,792 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,793 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:07,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,792 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,793 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:07,793 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,793 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:07,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,793 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,793 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,793 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,793 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,793 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,893 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:07,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:07,905 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:07,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:07,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,906 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,906 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,906 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 16:56:07,906 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 16:56:07,906 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,906 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,908 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,908 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,908 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,908 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,908 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,908 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,913 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,913 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,913 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 16:56:07,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,914 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,914 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,914 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. started... 2023-02-13 16:56:07,914 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 13acdfb7d6e238f0eec569bef8b8af44 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:07,916 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,917 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,917 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,917 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,917 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,917 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,930 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/fd73e6c0208342128d1fe4ea69471108 2023-02-13 16:56:07,942 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/35d006360296410fb3674e138216a44d 2023-02-13 16:56:07,945 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/fd73e6c0208342128d1fe4ea69471108 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/fd73e6c0208342128d1fe4ea69471108 2023-02-13 16:56:07,948 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/fd73e6c0208342128d1fe4ea69471108, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:56:07,949 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/35d006360296410fb3674e138216a44d as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/35d006360296410fb3674e138216a44d 2023-02-13 16:56:07,951 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/35d006360296410fb3674e138216a44d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:56:07,952 INFO [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 13acdfb7d6e238f0eec569bef8b8af44 in 38ms, sequenceid=71, compaction requested=true 2023-02-13 16:56:07,952 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 13acdfb7d6e238f0eec569bef8b8af44: 2023-02-13 16:56:07,952 DEBUG [rs(jenkins-hbase4.apache.org,34717,1676307263076)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:07,952 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:07,953 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:07,953 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,953 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 16:56:07,953 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 16:56:07,954 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,954 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,955 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,955 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 16:56:07,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@36b85cd2[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:07,956 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 16:56:07,957 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,967 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,967 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:07,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,967 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,967 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,967 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:07,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:07,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,967 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 16:56:07,968 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 16:56:07,968 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:07,969 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,969 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,969 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,969 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,969 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,969 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,970 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 16:56:07,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,970 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 16:56:07,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34717,1676307263076': 2023-02-13 16:56:07,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 16:56:07,973 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 16:56:07,973 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:07,973 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:07,973 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,973 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:07,975 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,975 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,975 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,975 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,975 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,975 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,976 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,975 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,976 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,976 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,978 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,978 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,978 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 16:56:07,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,982 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,982 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:07,983 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:07,983 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:07,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,983 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,983 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:07,983 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:07,983 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,983 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 16:56:07,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:08,083 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:08,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:08,091 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:08,091 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=13acdfb7d6e238f0eec569bef8b8af44, daughterA=aed0dd6f69a515c9a66e93b78ed8a70c, daughterB=86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,092 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=13acdfb7d6e238f0eec569bef8b8af44, daughterA=aed0dd6f69a515c9a66e93b78ed8a70c, daughterB=86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,093 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=13acdfb7d6e238f0eec569bef8b8af44, daughterA=aed0dd6f69a515c9a66e93b78ed8a70c, daughterB=86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,093 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=13acdfb7d6e238f0eec569bef8b8af44, daughterA=aed0dd6f69a515c9a66e93b78ed8a70c, daughterB=86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=13acdfb7d6e238f0eec569bef8b8af44, UNASSIGN}] 2023-02-13 16:56:08,095 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=13acdfb7d6e238f0eec569bef8b8af44, UNASSIGN 2023-02-13 16:56:08,095 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=13acdfb7d6e238f0eec569bef8b8af44, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:08,095 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307368095"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307368095"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307368095"}]},"ts":"1676307368095"} 2023-02-13 16:56:08,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 13acdfb7d6e238f0eec569bef8b8af44, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:56:08,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 13acdfb7d6e238f0eec569bef8b8af44, disabling compactions & flushes 2023-02-13 16:56:08,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:08,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:08,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. after waiting 0 ms 2023-02-13 16:56:08,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:08,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 13acdfb7d6e238f0eec569bef8b8af44 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 16:56:08,259 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/19aa9ed6c1684652816ed96cf6009baf 2023-02-13 16:56:08,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/6192587795604fdeb1f3675b16b93c7a 2023-02-13 16:56:08,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf1/19aa9ed6c1684652816ed96cf6009baf as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/19aa9ed6c1684652816ed96cf6009baf 2023-02-13 16:56:08,275 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/19aa9ed6c1684652816ed96cf6009baf, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:56:08,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.tmp/cf2/6192587795604fdeb1f3675b16b93c7a as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/6192587795604fdeb1f3675b16b93c7a 2023-02-13 16:56:08,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/6192587795604fdeb1f3675b16b93c7a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:56:08,279 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 13acdfb7d6e238f0eec569bef8b8af44 in 31ms, sequenceid=82, compaction requested=true 2023-02-13 16:56:08,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 16:56:08,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:08,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. 2023-02-13 16:56:08,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 13acdfb7d6e238f0eec569bef8b8af44: Waiting for close lock at 1676307368248Running coprocessor pre-close hooks at 1676307368248Disabling compacts and flushes for region at 1676307368248Disabling writes for close at 1676307368248Obtaining lock to block concurrent updates at 1676307368248Preparing flush snapshotting stores in 13acdfb7d6e238f0eec569bef8b8af44 at 1676307368248Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676307368248Flushing stores of testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. at 1676307368249 (+1 ms)Flushing 13acdfb7d6e238f0eec569bef8b8af44/cf1: creating writer at 1676307368251 (+2 ms)Flushing 13acdfb7d6e238f0eec569bef8b8af44/cf1: appending metadata at 1676307368253 (+2 ms)Flushing 13acdfb7d6e238f0eec569bef8b8af44/cf1: closing flushed file at 1676307368253Flushing 13acdfb7d6e238f0eec569bef8b8af44/cf2: creating writer at 1676307368262 (+9 ms)Flushing 13acdfb7d6e238f0eec569bef8b8af44/cf2: appending metadata at 1676307368263 (+1 ms)Flushing 13acdfb7d6e238f0eec569bef8b8af44/cf2: closing flushed file at 1676307368263Flushing 13acdfb7d6e238f0eec569bef8b8af44/cf1: reopening flushed file at 1676307368272 (+9 ms)Flushing 13acdfb7d6e238f0eec569bef8b8af44/cf2: reopening flushed file at 1676307368276 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 13acdfb7d6e238f0eec569bef8b8af44 in 31ms, sequenceid=82, compaction requested=true at 1676307368279 (+3 ms)Writing region close event to WAL at 1676307368281 (+2 ms)Running coprocessor post-close hooks at 1676307368284 (+3 ms)Closed at 1676307368284 2023-02-13 16:56:08,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,286 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=13acdfb7d6e238f0eec569bef8b8af44, regionState=CLOSED 2023-02-13 16:56:08,286 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307368285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307368285"}]},"ts":"1676307368285"} 2023-02-13 16:56:08,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-13 16:56:08,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 13acdfb7d6e238f0eec569bef8b8af44, server=jenkins-hbase4.apache.org,34717,1676307263076 in 191 msec 2023-02-13 16:56:08,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-13 16:56:08,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=13acdfb7d6e238f0eec569bef8b8af44, UNASSIGN in 194 msec 2023-02-13 16:56:08,298 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=13acdfb7d6e238f0eec569bef8b8af44, threads=14 2023-02-13 16:56:08,298 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/045baaca582b44d8ac74ddd767904a55 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,298 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/35d006360296410fb3674e138216a44d for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,298 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/59834d280d9d4e718b6ff7d9263dc9c7 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,298 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/6192587795604fdeb1f3675b16b93c7a for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,298 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/8d6b0bd20e724c1f8235f83db035462d for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,300 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/c50cf8f9428a4434bcacf9a410328542 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,300 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/d2d8291a943c4081976c19d4810e4f65 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,300 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/19aa9ed6c1684652816ed96cf6009baf for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,304 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/1d59cc3113164feab2f0bfff49401662 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,308 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/8635cc636eae4d6681e3dbffd4306864 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,308 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/982408db99074b45b40af2e2f2c24dfa for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,308 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/c945d8626f904fc485dbb9d4f38a6d36 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,312 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/d9ffd4875ad34a8da2b4a331a59d7a02 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,312 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/fd73e6c0208342128d1fe4ea69471108 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,315 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/19aa9ed6c1684652816ed96cf6009baf, top=false 2023-02-13 16:56:08,315 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/6192587795604fdeb1f3675b16b93c7a, top=false 2023-02-13 16:56:08,321 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/8635cc636eae4d6681e3dbffd4306864, top=false 2023-02-13 16:56:08,322 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/982408db99074b45b40af2e2f2c24dfa, top=false 2023-02-13 16:56:08,323 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/1d59cc3113164feab2f0bfff49401662, top=false 2023-02-13 16:56:08,324 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/c50cf8f9428a4434bcacf9a410328542, top=false 2023-02-13 16:56:08,325 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/8d6b0bd20e724c1f8235f83db035462d, top=false 2023-02-13 16:56:08,325 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/c945d8626f904fc485dbb9d4f38a6d36, top=false 2023-02-13 16:56:08,325 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/fd73e6c0208342128d1fe4ea69471108, top=false 2023-02-13 16:56:08,326 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/59834d280d9d4e718b6ff7d9263dc9c7, top=false 2023-02-13 16:56:08,326 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/d9ffd4875ad34a8da2b4a331a59d7a02, top=false 2023-02-13 16:56:08,326 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/35d006360296410fb3674e138216a44d, top=false 2023-02-13 16:56:08,327 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/045baaca582b44d8ac74ddd767904a55, top=false 2023-02-13 16:56:08,327 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/d2d8291a943c4081976c19d4810e4f65, top=false 2023-02-13 16:56:08,338 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-6192587795604fdeb1f3675b16b93c7a for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,339 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-1d59cc3113164feab2f0bfff49401662 for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,339 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-19aa9ed6c1684652816ed96cf6009baf for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,340 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-59834d280d9d4e718b6ff7d9263dc9c7 for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,341 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-982408db99074b45b40af2e2f2c24dfa for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,341 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8d6b0bd20e724c1f8235f83db035462d for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,342 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d9ffd4875ad34a8da2b4a331a59d7a02 for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,342 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-fd73e6c0208342128d1fe4ea69471108 for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,342 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d2d8291a943c4081976c19d4810e4f65 for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,343 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/6192587795604fdeb1f3675b16b93c7a for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,343 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/19aa9ed6c1684652816ed96cf6009baf for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,344 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-045baaca582b44d8ac74ddd767904a55 for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,344 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/1d59cc3113164feab2f0bfff49401662 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,345 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c50cf8f9428a4434bcacf9a410328542 for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,348 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8635cc636eae4d6681e3dbffd4306864 for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,348 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/982408db99074b45b40af2e2f2c24dfa for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,348 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/d9ffd4875ad34a8da2b4a331a59d7a02 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,349 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/59834d280d9d4e718b6ff7d9263dc9c7 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,350 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/d2d8291a943c4081976c19d4810e4f65 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,353 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/8d6b0bd20e724c1f8235f83db035462d for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,353 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-35d006360296410fb3674e138216a44d for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,353 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/.splits/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c945d8626f904fc485dbb9d4f38a6d36 for child: aed0dd6f69a515c9a66e93b78ed8a70c, parent: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,354 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/fd73e6c0208342128d1fe4ea69471108 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,356 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/c50cf8f9428a4434bcacf9a410328542 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,356 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/8635cc636eae4d6681e3dbffd4306864 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,356 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/045baaca582b44d8ac74ddd767904a55 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,357 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/35d006360296410fb3674e138216a44d for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,357 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/c945d8626f904fc485dbb9d4f38a6d36 for region: 13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:08,358 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 13acdfb7d6e238f0eec569bef8b8af44 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-13 16:56:08,385 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:56:08,387 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:56:08,388 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676307368388"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676307368388"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676307368388"}]},"ts":"1676307368388"} 2023-02-13 16:56:08,388 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307368388"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307368388"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307368388"}]},"ts":"1676307368388"} 2023-02-13 16:56:08,388 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307368388"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307368388"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307368388"}]},"ts":"1676307368388"} 2023-02-13 16:56:08,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=aed0dd6f69a515c9a66e93b78ed8a70c, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=86569665c1c8ddfe3e395139a2efd6a8, ASSIGN}] 2023-02-13 16:56:08,392 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=86569665c1c8ddfe3e395139a2efd6a8, ASSIGN 2023-02-13 16:56:08,392 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=86569665c1c8ddfe3e395139a2efd6a8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:56:08,393 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=aed0dd6f69a515c9a66e93b78ed8a70c, ASSIGN 2023-02-13 16:56:08,393 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=aed0dd6f69a515c9a66e93b78ed8a70c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34717,1676307263076; forceNewPlan=false, retain=false 2023-02-13 16:56:08,530 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:56:08,542 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 16:56:08,544 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=aed0dd6f69a515c9a66e93b78ed8a70c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:08,544 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307368544"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307368544"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307368544"}]},"ts":"1676307368544"} 2023-02-13 16:56:08,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure aed0dd6f69a515c9a66e93b78ed8a70c, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:56:08,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=86569665c1c8ddfe3e395139a2efd6a8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:08,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307368548"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307368548"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307368548"}]},"ts":"1676307368548"} 2023-02-13 16:56:08,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure 86569665c1c8ddfe3e395139a2efd6a8, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:56:08,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. 2023-02-13 16:56:08,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => aed0dd6f69a515c9a66e93b78ed8a70c, NAME => 'testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.', STARTKEY => '', ENDKEY => '71'} 2023-02-13 16:56:08,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:56:08,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:08,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:08,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:08,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:08,701 INFO [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:08,702 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1 2023-02-13 16:56:08,702 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1 2023-02-13 16:56:08,702 INFO [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aed0dd6f69a515c9a66e93b78ed8a70c columnFamilyName cf1 2023-02-13 16:56:08,707 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-19aa9ed6c1684652816ed96cf6009baf 2023-02-13 16:56:08,709 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-1d59cc3113164feab2f0bfff49401662 2023-02-13 16:56:08,711 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8635cc636eae4d6681e3dbffd4306864 2023-02-13 16:56:08,714 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-982408db99074b45b40af2e2f2c24dfa 2023-02-13 16:56:08,716 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c945d8626f904fc485dbb9d4f38a6d36 2023-02-13 16:56:08,718 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d9ffd4875ad34a8da2b4a331a59d7a02 2023-02-13 16:56:08,720 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-fd73e6c0208342128d1fe4ea69471108 2023-02-13 16:56:08,721 INFO [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(310): Store=aed0dd6f69a515c9a66e93b78ed8a70c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:08,721 INFO [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:08,721 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2 2023-02-13 16:56:08,721 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2 2023-02-13 16:56:08,722 INFO [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aed0dd6f69a515c9a66e93b78ed8a70c columnFamilyName cf2 2023-02-13 16:56:08,725 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-045baaca582b44d8ac74ddd767904a55 2023-02-13 16:56:08,727 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-35d006360296410fb3674e138216a44d 2023-02-13 16:56:08,730 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-59834d280d9d4e718b6ff7d9263dc9c7 2023-02-13 16:56:08,732 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-6192587795604fdeb1f3675b16b93c7a 2023-02-13 16:56:08,735 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8d6b0bd20e724c1f8235f83db035462d 2023-02-13 16:56:08,737 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c50cf8f9428a4434bcacf9a410328542 2023-02-13 16:56:08,740 DEBUG [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d2d8291a943c4081976c19d4810e4f65 2023-02-13 16:56:08,740 INFO [StoreOpener-aed0dd6f69a515c9a66e93b78ed8a70c-1] regionserver.HStore(310): Store=aed0dd6f69a515c9a66e93b78ed8a70c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:08,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:08,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:08,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:08,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:08,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened aed0dd6f69a515c9a66e93b78ed8a70c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62676660, jitterRate=-0.06604498624801636}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:08,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for aed0dd6f69a515c9a66e93b78ed8a70c: Running coprocessor pre-open hook at 1676307368700Writing region info on filesystem at 1676307368700Initializing all the Stores at 1676307368701 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307368701Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307368701Cleaning up temporary data from old regions at 1676307368742 (+41 ms)Cleaning up detritus from prior splits at 1676307368742Running coprocessor post-open hooks at 1676307368745 (+3 ms)Region opened successfully at 1676307368745 2023-02-13 16:56:08,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c., pid=133, masterSystemTime=1676307368697 2023-02-13 16:56:08,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:08,747 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:08,747 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:08,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:08,747 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 16:56:08,747 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 16:56:08,747 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1912): aed0dd6f69a515c9a66e93b78ed8a70c/cf1 is initiating minor compaction (all files) 2023-02-13 16:56:08,747 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1912): aed0dd6f69a515c9a66e93b78ed8a70c/cf2 is initiating minor compaction (all files) 2023-02-13 16:56:08,747 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of aed0dd6f69a515c9a66e93b78ed8a70c/cf1 in testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. 2023-02-13 16:56:08,747 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2258): Starting compaction of aed0dd6f69a515c9a66e93b78ed8a70c/cf2 in testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. 2023-02-13 16:56:08,747 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c945d8626f904fc485dbb9d4f38a6d36, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d9ffd4875ad34a8da2b4a331a59d7a02, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-1d59cc3113164feab2f0bfff49401662, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-982408db99074b45b40af2e2f2c24dfa, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8635cc636eae4d6681e3dbffd4306864, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-fd73e6c0208342128d1fe4ea69471108, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-19aa9ed6c1684652816ed96cf6009baf] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/.tmp, totalSize=34.9 K 2023-02-13 16:56:08,747 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d2d8291a943c4081976c19d4810e4f65, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c50cf8f9428a4434bcacf9a410328542, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-045baaca582b44d8ac74ddd767904a55, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-59834d280d9d4e718b6ff7d9263dc9c7, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8d6b0bd20e724c1f8235f83db035462d, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-35d006360296410fb3674e138216a44d, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-6192587795604fdeb1f3675b16b93c7a] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/.tmp, totalSize=34.9 K 2023-02-13 16:56:08,747 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c945d8626f904fc485dbb9d4f38a6d36, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307367050 2023-02-13 16:56:08,748 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d2d8291a943c4081976c19d4810e4f65, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307367050 2023-02-13 16:56:08,748 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d9ffd4875ad34a8da2b4a331a59d7a02, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307367215 2023-02-13 16:56:08,748 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c50cf8f9428a4434bcacf9a410328542, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307367215 2023-02-13 16:56:08,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. 2023-02-13 16:56:08,748 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-1d59cc3113164feab2f0bfff49401662, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307367381 2023-02-13 16:56:08,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. 2023-02-13 16:56:08,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8. 2023-02-13 16:56:08,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 86569665c1c8ddfe3e395139a2efd6a8, NAME => 'testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8.', STARTKEY => '71', ENDKEY => ''} 2023-02-13 16:56:08,749 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-045baaca582b44d8ac74ddd767904a55, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307367381 2023-02-13 16:56:08,749 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=aed0dd6f69a515c9a66e93b78ed8a70c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:08,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:56:08,749 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-982408db99074b45b40af2e2f2c24dfa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307367557 2023-02-13 16:56:08,749 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307368749"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307368749"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307368749"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307368749"}]},"ts":"1676307368749"} 2023-02-13 16:56:08,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,749 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-59834d280d9d4e718b6ff7d9263dc9c7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307367557 2023-02-13 16:56:08,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:08,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,749 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8635cc636eae4d6681e3dbffd4306864, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307367727 2023-02-13 16:56:08,750 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8d6b0bd20e724c1f8235f83db035462d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307367727 2023-02-13 16:56:08,750 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-fd73e6c0208342128d1fe4ea69471108, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307367895 2023-02-13 16:56:08,750 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-35d006360296410fb3674e138216a44d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307367895 2023-02-13 16:56:08,750 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-19aa9ed6c1684652816ed96cf6009baf, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307368085 2023-02-13 16:56:08,751 INFO [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,751 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-6192587795604fdeb1f3675b16b93c7a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307368085 2023-02-13 16:56:08,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-02-13 16:56:08,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure aed0dd6f69a515c9a66e93b78ed8a70c, server=jenkins-hbase4.apache.org,34717,1676307263076 in 205 msec 2023-02-13 16:56:08,754 DEBUG [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/cf1 2023-02-13 16:56:08,754 DEBUG [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/cf1 2023-02-13 16:56:08,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=aed0dd6f69a515c9a66e93b78ed8a70c, ASSIGN in 363 msec 2023-02-13 16:56:08,755 INFO [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86569665c1c8ddfe3e395139a2efd6a8 columnFamilyName cf1 2023-02-13 16:56:08,755 INFO [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] regionserver.HStore(310): Store=86569665c1c8ddfe3e395139a2efd6a8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:08,756 INFO [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,757 DEBUG [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/cf2 2023-02-13 16:56:08,757 DEBUG [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/cf2 2023-02-13 16:56:08,757 INFO [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86569665c1c8ddfe3e395139a2efd6a8 columnFamilyName cf2 2023-02-13 16:56:08,758 INFO [StoreOpener-86569665c1c8ddfe3e395139a2efd6a8-1] regionserver.HStore(310): Store=86569665c1c8ddfe3e395139a2efd6a8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:08,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:08,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:08,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 86569665c1c8ddfe3e395139a2efd6a8; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67799243, jitterRate=0.010287448763847351}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:08,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 86569665c1c8ddfe3e395139a2efd6a8: Running coprocessor pre-open hook at 1676307368749Writing region info on filesystem at 1676307368749Initializing all the Stores at 1676307368750 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307368750Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307368750Cleaning up temporary data from old regions at 1676307368760 (+10 ms)Cleaning up detritus from prior splits at 1676307368761 (+1 ms)Running coprocessor post-open hooks at 1676307368764 (+3 ms)Region opened successfully at 1676307368764 2023-02-13 16:56:08,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8., pid=134, masterSystemTime=1676307368697 2023-02-13 16:56:08,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8. 2023-02-13 16:56:08,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8. 2023-02-13 16:56:08,766 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=86569665c1c8ddfe3e395139a2efd6a8, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:08,766 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307368766"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307368766"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307368766"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307368766"}]},"ts":"1676307368766"} 2023-02-13 16:56:08,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-02-13 16:56:08,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure 86569665c1c8ddfe3e395139a2efd6a8, server=jenkins-hbase4.apache.org,34717,1676307263076 in 218 msec 2023-02-13 16:56:08,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-13 16:56:08,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=86569665c1c8ddfe3e395139a2efd6a8, ASSIGN in 378 msec 2023-02-13 16:56:08,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=13acdfb7d6e238f0eec569bef8b8af44, daughterA=aed0dd6f69a515c9a66e93b78ed8a70c, daughterB=86569665c1c8ddfe3e395139a2efd6a8 in 680 msec 2023-02-13 16:56:08,788 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] throttle.PressureAwareThroughputController(145): aed0dd6f69a515c9a66e93b78ed8a70c#cf2#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:08,789 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] throttle.PressureAwareThroughputController(145): aed0dd6f69a515c9a66e93b78ed8a70c#cf1#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:08,806 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/.tmp/cf1/f9f44bd5dfe944bcbb143096043d7985 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/f9f44bd5dfe944bcbb143096043d7985 2023-02-13 16:56:08,806 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/.tmp/cf2/372a1bd2b3f0483dbebe671aa94f31a1 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/372a1bd2b3f0483dbebe671aa94f31a1 2023-02-13 16:56:08,810 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in aed0dd6f69a515c9a66e93b78ed8a70c/cf1 of aed0dd6f69a515c9a66e93b78ed8a70c into f9f44bd5dfe944bcbb143096043d7985(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:08,810 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in aed0dd6f69a515c9a66e93b78ed8a70c/cf2 of aed0dd6f69a515c9a66e93b78ed8a70c into 372a1bd2b3f0483dbebe671aa94f31a1(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:08,810 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for aed0dd6f69a515c9a66e93b78ed8a70c: 2023-02-13 16:56:08,811 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for aed0dd6f69a515c9a66e93b78ed8a70c: 2023-02-13 16:56:08,811 INFO [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c., storeName=aed0dd6f69a515c9a66e93b78ed8a70c/cf1, priority=9, startTime=1676307368746; duration=0sec 2023-02-13 16:56:08,811 INFO [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c., storeName=aed0dd6f69a515c9a66e93b78ed8a70c/cf2, priority=9, startTime=1676307368747; duration=0sec 2023-02-13 16:56:08,811 DEBUG [RS:2;jenkins-hbase4:34717-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:08,811 DEBUG [RS:2;jenkins-hbase4:34717-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:08,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34717] ipc.CallRunner(144): callId: 1027 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:48874 deadline: 1676307428845, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44. is not online on jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:08,955 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-13 16:56:08,958 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:56:08,960 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-13 16:56:08,960 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-13 16:56:08,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 16:56:08,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 16:56:08,963 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307368963"}]},"ts":"1676307368963"} 2023-02-13 16:56:08,964 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-13 16:56:08,965 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-13 16:56:08,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=aed0dd6f69a515c9a66e93b78ed8a70c, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=86569665c1c8ddfe3e395139a2efd6a8, UNASSIGN}] 2023-02-13 16:56:08,966 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=86569665c1c8ddfe3e395139a2efd6a8, UNASSIGN 2023-02-13 16:56:08,967 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=86569665c1c8ddfe3e395139a2efd6a8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:08,967 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307368967"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307368967"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307368967"}]},"ts":"1676307368967"} 2023-02-13 16:56:08,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 86569665c1c8ddfe3e395139a2efd6a8, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:56:08,970 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=aed0dd6f69a515c9a66e93b78ed8a70c, UNASSIGN 2023-02-13 16:56:08,970 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=aed0dd6f69a515c9a66e93b78ed8a70c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:08,970 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307368970"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307368970"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307368970"}]},"ts":"1676307368970"} 2023-02-13 16:56:08,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure aed0dd6f69a515c9a66e93b78ed8a70c, server=jenkins-hbase4.apache.org,34717,1676307263076}] 2023-02-13 16:56:09,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 16:56:09,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:09,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 86569665c1c8ddfe3e395139a2efd6a8, disabling compactions & flushes 2023-02-13 16:56:09,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8. 2023-02-13 16:56:09,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8. 2023-02-13 16:56:09,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8. after waiting 0 ms 2023-02-13 16:56:09,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8. 2023-02-13 16:56:09,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-13 16:56:09,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:09,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8. 2023-02-13 16:56:09,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 86569665c1c8ddfe3e395139a2efd6a8: Waiting for close lock at 1676307369120Running coprocessor pre-close hooks at 1676307369120Disabling compacts and flushes for region at 1676307369120Disabling writes for close at 1676307369121 (+1 ms)Writing region close event to WAL at 1676307369121Running coprocessor post-close hooks at 1676307369124 (+3 ms)Closed at 1676307369124 2023-02-13 16:56:09,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:09,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:09,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing aed0dd6f69a515c9a66e93b78ed8a70c, disabling compactions & flushes 2023-02-13 16:56:09,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. 2023-02-13 16:56:09,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. 2023-02-13 16:56:09,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. after waiting 0 ms 2023-02-13 16:56:09,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. 2023-02-13 16:56:09,126 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=86569665c1c8ddfe3e395139a2efd6a8, regionState=CLOSED 2023-02-13 16:56:09,126 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307369126"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307369126"}]},"ts":"1676307369126"} 2023-02-13 16:56:09,126 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c945d8626f904fc485dbb9d4f38a6d36, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d9ffd4875ad34a8da2b4a331a59d7a02, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-1d59cc3113164feab2f0bfff49401662, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-982408db99074b45b40af2e2f2c24dfa, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8635cc636eae4d6681e3dbffd4306864, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-fd73e6c0208342128d1fe4ea69471108, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-19aa9ed6c1684652816ed96cf6009baf] to archive 2023-02-13 16:56:09,127 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:09,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-13 16:56:09,129 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c945d8626f904fc485dbb9d4f38a6d36 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c945d8626f904fc485dbb9d4f38a6d36 2023-02-13 16:56:09,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 86569665c1c8ddfe3e395139a2efd6a8, server=jenkins-hbase4.apache.org,34717,1676307263076 in 159 msec 2023-02-13 16:56:09,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=86569665c1c8ddfe3e395139a2efd6a8, UNASSIGN in 163 msec 2023-02-13 16:56:09,130 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d9ffd4875ad34a8da2b4a331a59d7a02 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d9ffd4875ad34a8da2b4a331a59d7a02 2023-02-13 16:56:09,131 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-1d59cc3113164feab2f0bfff49401662 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-1d59cc3113164feab2f0bfff49401662 2023-02-13 16:56:09,132 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-982408db99074b45b40af2e2f2c24dfa to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-982408db99074b45b40af2e2f2c24dfa 2023-02-13 16:56:09,133 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8635cc636eae4d6681e3dbffd4306864 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8635cc636eae4d6681e3dbffd4306864 2023-02-13 16:56:09,134 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-fd73e6c0208342128d1fe4ea69471108 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-fd73e6c0208342128d1fe4ea69471108 2023-02-13 16:56:09,135 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-19aa9ed6c1684652816ed96cf6009baf to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-19aa9ed6c1684652816ed96cf6009baf 2023-02-13 16:56:09,136 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d2d8291a943c4081976c19d4810e4f65, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c50cf8f9428a4434bcacf9a410328542, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-045baaca582b44d8ac74ddd767904a55, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-59834d280d9d4e718b6ff7d9263dc9c7, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8d6b0bd20e724c1f8235f83db035462d, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-35d006360296410fb3674e138216a44d, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-6192587795604fdeb1f3675b16b93c7a] to archive 2023-02-13 16:56:09,137 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:09,138 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d2d8291a943c4081976c19d4810e4f65 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-d2d8291a943c4081976c19d4810e4f65 2023-02-13 16:56:09,139 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c50cf8f9428a4434bcacf9a410328542 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-c50cf8f9428a4434bcacf9a410328542 2023-02-13 16:56:09,140 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-045baaca582b44d8ac74ddd767904a55 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-045baaca582b44d8ac74ddd767904a55 2023-02-13 16:56:09,140 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-59834d280d9d4e718b6ff7d9263dc9c7 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-59834d280d9d4e718b6ff7d9263dc9c7 2023-02-13 16:56:09,141 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8d6b0bd20e724c1f8235f83db035462d to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-8d6b0bd20e724c1f8235f83db035462d 2023-02-13 16:56:09,142 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-35d006360296410fb3674e138216a44d to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-35d006360296410fb3674e138216a44d 2023-02-13 16:56:09,143 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-6192587795604fdeb1f3675b16b93c7a to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/testSplitTableRegionEmptyDaughter=13acdfb7d6e238f0eec569bef8b8af44-6192587795604fdeb1f3675b16b93c7a 2023-02-13 16:56:09,146 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:56:09,146 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:09,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c. 2023-02-13 16:56:09,146 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for aed0dd6f69a515c9a66e93b78ed8a70c: Waiting for close lock at 1676307369126Running coprocessor pre-close hooks at 1676307369126Disabling compacts and flushes for region at 1676307369126Disabling writes for close at 1676307369126Writing region close event to WAL at 1676307369144 (+18 ms)Running coprocessor post-close hooks at 1676307369146 (+2 ms)Closed at 1676307369146 2023-02-13 16:56:09,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:09,148 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=aed0dd6f69a515c9a66e93b78ed8a70c, regionState=CLOSED 2023-02-13 16:56:09,148 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307369148"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307369148"}]},"ts":"1676307369148"} 2023-02-13 16:56:09,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-13 16:56:09,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure aed0dd6f69a515c9a66e93b78ed8a70c, server=jenkins-hbase4.apache.org,34717,1676307263076 in 178 msec 2023-02-13 16:56:09,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-02-13 16:56:09,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=aed0dd6f69a515c9a66e93b78ed8a70c, UNASSIGN in 184 msec 2023-02-13 16:56:09,151 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307369151"}]},"ts":"1676307369151"} 2023-02-13 16:56:09,152 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-13 16:56:09,154 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-13 16:56:09,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 194 msec 2023-02-13 16:56:09,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 16:56:09,265 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-13 16:56:09,265 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-13 16:56:09,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 16:56:09,268 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 16:56:09,268 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 16:56:09,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 16:56:09,272 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:09,272 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:09,272 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:09,274 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/recovered.edits] 2023-02-13 16:56:09,275 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/recovered.edits] 2023-02-13 16:56:09,275 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/recovered.edits] 2023-02-13 16:56:09,278 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/f9f44bd5dfe944bcbb143096043d7985 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf1/f9f44bd5dfe944bcbb143096043d7985 2023-02-13 16:56:09,278 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/19aa9ed6c1684652816ed96cf6009baf to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/19aa9ed6c1684652816ed96cf6009baf 2023-02-13 16:56:09,280 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/1d59cc3113164feab2f0bfff49401662 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/1d59cc3113164feab2f0bfff49401662 2023-02-13 16:56:09,280 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/recovered.edits/88.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8/recovered.edits/88.seqid 2023-02-13 16:56:09,280 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/372a1bd2b3f0483dbebe671aa94f31a1 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/cf2/372a1bd2b3f0483dbebe671aa94f31a1 2023-02-13 16:56:09,281 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/86569665c1c8ddfe3e395139a2efd6a8 2023-02-13 16:56:09,281 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/8635cc636eae4d6681e3dbffd4306864 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/8635cc636eae4d6681e3dbffd4306864 2023-02-13 16:56:09,283 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/982408db99074b45b40af2e2f2c24dfa to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/982408db99074b45b40af2e2f2c24dfa 2023-02-13 16:56:09,283 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c/recovered.edits/90.seqid 2023-02-13 16:56:09,284 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/aed0dd6f69a515c9a66e93b78ed8a70c 2023-02-13 16:56:09,284 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/c945d8626f904fc485dbb9d4f38a6d36 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/c945d8626f904fc485dbb9d4f38a6d36 2023-02-13 16:56:09,285 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/d9ffd4875ad34a8da2b4a331a59d7a02 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/d9ffd4875ad34a8da2b4a331a59d7a02 2023-02-13 16:56:09,286 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/fd73e6c0208342128d1fe4ea69471108 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf1/fd73e6c0208342128d1fe4ea69471108 2023-02-13 16:56:09,289 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/045baaca582b44d8ac74ddd767904a55 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/045baaca582b44d8ac74ddd767904a55 2023-02-13 16:56:09,290 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/35d006360296410fb3674e138216a44d to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/35d006360296410fb3674e138216a44d 2023-02-13 16:56:09,292 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/59834d280d9d4e718b6ff7d9263dc9c7 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/59834d280d9d4e718b6ff7d9263dc9c7 2023-02-13 16:56:09,293 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/6192587795604fdeb1f3675b16b93c7a to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/6192587795604fdeb1f3675b16b93c7a 2023-02-13 16:56:09,294 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/8d6b0bd20e724c1f8235f83db035462d to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/8d6b0bd20e724c1f8235f83db035462d 2023-02-13 16:56:09,296 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/c50cf8f9428a4434bcacf9a410328542 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/c50cf8f9428a4434bcacf9a410328542 2023-02-13 16:56:09,297 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/d2d8291a943c4081976c19d4810e4f65 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/cf2/d2d8291a943c4081976c19d4810e4f65 2023-02-13 16:56:09,300 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/recovered.edits/85.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44/recovered.edits/85.seqid 2023-02-13 16:56:09,300 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegionEmptyDaughter/13acdfb7d6e238f0eec569bef8b8af44 2023-02-13 16:56:09,300 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 16:56:09,303 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 16:56:09,304 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-13 16:56:09,306 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-13 16:56:09,307 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 16:56:09,307 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-13 16:56:09,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307369307"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:09,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307369307"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:09,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307369307"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:09,309 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 16:56:09,309 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 13acdfb7d6e238f0eec569bef8b8af44, NAME => 'testSplitTableRegionEmptyDaughter,,1676307366547.13acdfb7d6e238f0eec569bef8b8af44.', STARTKEY => '', ENDKEY => ''}, {ENCODED => aed0dd6f69a515c9a66e93b78ed8a70c, NAME => 'testSplitTableRegionEmptyDaughter,,1676307368091.aed0dd6f69a515c9a66e93b78ed8a70c.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 86569665c1c8ddfe3e395139a2efd6a8, NAME => 'testSplitTableRegionEmptyDaughter,71,1676307368091.86569665c1c8ddfe3e395139a2efd6a8.', STARTKEY => '71', ENDKEY => ''}] 2023-02-13 16:56:09,309 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-13 16:56:09,309 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307369309"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:09,310 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-13 16:56:09,312 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 16:56:09,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 48 msec 2023-02-13 16:56:09,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 16:56:09,370 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-13 16:56:09,383 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 375), OpenFileDescriptor=832 (was 832), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=99 (was 108), ProcessCount=168 (was 168), AvailableMemoryMB=1037 (was 1040) 2023-02-13 16:56:09,397 INFO [Listener at localhost/38547] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=832, MaxFileDescriptor=60000, SystemLoadAverage=99, ProcessCount=168, AvailableMemoryMB=1037 2023-02-13 16:56:09,397 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:56:09,401 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 16:56:09,402 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 16:56:09,403 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-13 16:56:09,404 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:56:09,405 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:56:09,406 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,406 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3 empty. 2023-02-13 16:56:09,407 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,407 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 16:56:09,415 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-13 16:56:09,416 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1941e9caf41b5697594d4b5192c066a3, NAME => 'testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp 2023-02-13 16:56:09,430 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:09,430 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 1941e9caf41b5697594d4b5192c066a3, disabling compactions & flushes 2023-02-13 16:56:09,430 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:09,430 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:09,430 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. after waiting 0 ms 2023-02-13 16:56:09,430 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:09,430 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:09,430 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 1941e9caf41b5697594d4b5192c066a3: Waiting for close lock at 1676307369430Disabling compacts and flushes for region at 1676307369430Disabling writes for close at 1676307369430Writing region close event to WAL at 1676307369430Closed at 1676307369430 2023-02-13 16:56:09,433 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:56:09,433 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307369433"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307369433"}]},"ts":"1676307369433"} 2023-02-13 16:56:09,435 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:56:09,435 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:56:09,435 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307369435"}]},"ts":"1676307369435"} 2023-02-13 16:56:09,436 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-13 16:56:09,440 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 16:56:09,441 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 16:56:09,441 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 16:56:09,441 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 16:56:09,441 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 16:56:09,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=1941e9caf41b5697594d4b5192c066a3, ASSIGN}] 2023-02-13 16:56:09,442 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=1941e9caf41b5697594d4b5192c066a3, ASSIGN 2023-02-13 16:56:09,443 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=1941e9caf41b5697594d4b5192c066a3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38051,1676307263041; forceNewPlan=false, retain=false 2023-02-13 16:56:09,593 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 16:56:09,594 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=1941e9caf41b5697594d4b5192c066a3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,594 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307369594"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307369594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307369594"}]},"ts":"1676307369594"} 2023-02-13 16:56:09,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 1941e9caf41b5697594d4b5192c066a3, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:56:09,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:09,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1941e9caf41b5697594d4b5192c066a3, NAME => 'testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:56:09,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:56:09,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:09,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,754 INFO [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,755 DEBUG [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1 2023-02-13 16:56:09,755 DEBUG [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1 2023-02-13 16:56:09,755 INFO [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1941e9caf41b5697594d4b5192c066a3 columnFamilyName cf1 2023-02-13 16:56:09,756 INFO [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] regionserver.HStore(310): Store=1941e9caf41b5697594d4b5192c066a3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:09,756 INFO [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,757 DEBUG [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2 2023-02-13 16:56:09,757 DEBUG [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2 2023-02-13 16:56:09,757 INFO [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1941e9caf41b5697594d4b5192c066a3 columnFamilyName cf2 2023-02-13 16:56:09,758 INFO [StoreOpener-1941e9caf41b5697594d4b5192c066a3-1] regionserver.HStore(310): Store=1941e9caf41b5697594d4b5192c066a3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:09,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:09,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:09,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:56:09,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1941e9caf41b5697594d4b5192c066a3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66055805, jitterRate=-0.01569180190563202}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:09,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1941e9caf41b5697594d4b5192c066a3: Running coprocessor pre-open hook at 1676307369753Writing region info on filesystem at 1676307369753Initializing all the Stores at 1676307369754 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307369754Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307369754Cleaning up temporary data from old regions at 1676307369759 (+5 ms)Cleaning up detritus from prior splits at 1676307369760 (+1 ms)Running coprocessor post-open hooks at 1676307369763 (+3 ms)Region opened successfully at 1676307369763 2023-02-13 16:56:09,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3., pid=143, masterSystemTime=1676307369747 2023-02-13 16:56:09,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:09,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:09,765 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=1941e9caf41b5697594d4b5192c066a3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,765 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307369765"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307369765"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307369765"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307369765"}]},"ts":"1676307369765"} 2023-02-13 16:56:09,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-13 16:56:09,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 1941e9caf41b5697594d4b5192c066a3, server=jenkins-hbase4.apache.org,38051,1676307263041 in 170 msec 2023-02-13 16:56:09,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-13 16:56:09,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=1941e9caf41b5697594d4b5192c066a3, ASSIGN in 326 msec 2023-02-13 16:56:09,769 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:56:09,769 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307369769"}]},"ts":"1676307369769"} 2023-02-13 16:56:09,770 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-13 16:56:09,771 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:56:09,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 369 msec 2023-02-13 16:56:09,909 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:09,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 16:56:09,911 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 16:56:09,911 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:09,911 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 16:56:09,911 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 16:56:09,911 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,911 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:09,914 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:09,914 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:09,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:09,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:09,914 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:09,914 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:09,914 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,914 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:09,915 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:09,915 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:09,915 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:09,916 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:09,916 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:09,916 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. started... 2023-02-13 16:56:09,917 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1941e9caf41b5697594d4b5192c066a3 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 16:56:09,917 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,918 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,918 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,918 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:09,925 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/535e99663dd142b892e5f90f11f95da4 2023-02-13 16:56:09,938 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/c2dfe3394d6844b686516bd0ec17ba90 2023-02-13 16:56:09,941 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/535e99663dd142b892e5f90f11f95da4 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4 2023-02-13 16:56:09,944 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:56:09,945 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/c2dfe3394d6844b686516bd0ec17ba90 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90 2023-02-13 16:56:09,948 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:56:09,948 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 1941e9caf41b5697594d4b5192c066a3 in 31ms, sequenceid=6, compaction requested=false 2023-02-13 16:56:09,949 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-13 16:56:09,949 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1941e9caf41b5697594d4b5192c066a3: 2023-02-13 16:56:09,949 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:09,949 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:09,949 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:09,949 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,949 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:09,949 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:09,951 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,951 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:09,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:09,951 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,951 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:09,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:09,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:09,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:09,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:09,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:09,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:09,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 16:56:09,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ffc2db3[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:09,953 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 16:56:09,953 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,955 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,955 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:09,955 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:09,955 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:09,955 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,955 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:09,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,956 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:09,956 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:09,956 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:09,956 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:09,956 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:09,956 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:09,957 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:09,957 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:09,957 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:09,961 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:09,961 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:09,961 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,961 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:09,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:09,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:09,961 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:09,961 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:09,961 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:09,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:09,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:09,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:09,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:09,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:09,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:09,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:09,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:09,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:09,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:56:09,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 16:56:09,964 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 16:56:09,964 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:09,964 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:09,964 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 16:56:09,964 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:09,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:09,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:09,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:09,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,967 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:09,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:09,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:09,968 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:09,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:09,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:09,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:09,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:09,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:09,968 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:09,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:09,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:09,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:09,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:09,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:09,969 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:09,970 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:09,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,970 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:09,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:09,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:09,972 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:09,972 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:09,972 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:09,972 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:09,972 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:09,972 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 16:56:09,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:09,972 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:09,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:09,972 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:09,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:09,972 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:09,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:09,972 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:09,972 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:09,972 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:09,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:09,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:09,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:09,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:09,973 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,073 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:10,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:10,085 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:10,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:10,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 16:56:10,087 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 16:56:10,087 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,087 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 16:56:10,087 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 16:56:10,087 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,087 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,088 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,088 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,088 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,088 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,089 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,089 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:10,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,090 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,090 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,090 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,090 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,090 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,090 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,090 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,090 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,091 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,091 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,091 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,091 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,091 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,091 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:10,091 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:10,091 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,091 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. started... 2023-02-13 16:56:10,091 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1941e9caf41b5697594d4b5192c066a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:10,092 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,092 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,092 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,096 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,096 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,096 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,105 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/73582f3711d043a2b105067b15a3c302 2023-02-13 16:56:10,118 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/f54475d5c17e49ae93434e756aebed92 2023-02-13 16:56:10,122 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/73582f3711d043a2b105067b15a3c302 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302 2023-02-13 16:56:10,126 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:56:10,126 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/f54475d5c17e49ae93434e756aebed92 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92 2023-02-13 16:56:10,129 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 16:56:10,130 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1941e9caf41b5697594d4b5192c066a3 in 39ms, sequenceid=19, compaction requested=false 2023-02-13 16:56:10,130 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1941e9caf41b5697594d4b5192c066a3: 2023-02-13 16:56:10,130 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,130 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:10,130 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:10,130 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,130 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,130 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,132 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,132 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,133 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,133 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 16:56:10,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b320822[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:10,135 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 16:56:10,135 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,136 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,136 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,137 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,137 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,137 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,137 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,137 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,137 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:10,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,137 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:10,137 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,137 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:10,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,137 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,137 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,137 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:10,139 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,139 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,139 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,139 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,139 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,139 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,139 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,141 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,141 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,141 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:56:10,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 16:56:10,144 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 16:56:10,144 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:10,145 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:10,145 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 16:56:10,145 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:10,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,147 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,147 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,146 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,147 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,148 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,150 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,150 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,150 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,153 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 16:56:10,154 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:10,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,154 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:10,153 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,153 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,154 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:10,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,154 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,154 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,155 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,155 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,155 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,155 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,155 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,155 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,155 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,254 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:10,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:10,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:10,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:10,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 16:56:10,268 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 16:56:10,268 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,268 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 16:56:10,268 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 16:56:10,268 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,268 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,269 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,269 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,270 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:10,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,271 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,271 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,271 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,272 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,272 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,272 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,272 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,272 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,272 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,272 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:10,272 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:10,272 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,272 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. started... 2023-02-13 16:56:10,272 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1941e9caf41b5697594d4b5192c066a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:10,274 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,277 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,277 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,277 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,277 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,277 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,283 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/97320f225b0a4b14a5dfdeb01b068255 2023-02-13 16:56:10,297 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6 2023-02-13 16:56:10,303 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/97320f225b0a4b14a5dfdeb01b068255 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255 2023-02-13 16:56:10,307 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:56:10,308 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6 2023-02-13 16:56:10,311 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 16:56:10,312 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1941e9caf41b5697594d4b5192c066a3 in 40ms, sequenceid=32, compaction requested=true 2023-02-13 16:56:10,312 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1941e9caf41b5697594d4b5192c066a3: 2023-02-13 16:56:10,312 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,312 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:10,313 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:10,313 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,313 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,313 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,314 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,314 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,315 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,315 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 16:56:10,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44665732[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:10,318 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 16:56:10,318 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,319 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,319 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,319 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,319 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,320 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,320 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:10,320 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,320 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,320 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,320 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,320 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:10,320 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,320 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,320 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:10,322 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,322 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,322 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,322 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,323 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,323 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,324 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,324 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,324 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,324 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:56:10,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 16:56:10,328 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 16:56:10,329 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:10,329 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:10,329 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 16:56:10,329 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:10,330 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,330 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,330 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,331 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,330 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,330 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,331 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,331 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,332 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,332 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,334 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,335 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,335 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,339 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 16:56:10,339 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,339 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,340 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:10,339 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,340 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:10,340 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,340 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,340 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,340 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,341 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,341 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,341 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,440 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:10,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:10,451 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:10,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:10,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 16:56:10,453 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 16:56:10,453 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,453 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 16:56:10,453 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 16:56:10,453 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,453 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,455 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,455 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,455 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,455 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,455 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,456 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,456 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,457 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,457 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,457 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,457 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,457 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,457 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:10,457 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:10,458 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,458 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. started... 2023-02-13 16:56:10,458 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1941e9caf41b5697594d4b5192c066a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:10,459 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,459 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,459 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,459 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,459 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,459 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,470 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/800bb38182e44e1ba943f82908fc677e 2023-02-13 16:56:10,483 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/efe858801fd54c2286f6d3b6825da470 2023-02-13 16:56:10,487 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/800bb38182e44e1ba943f82908fc677e as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e 2023-02-13 16:56:10,490 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:56:10,490 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/efe858801fd54c2286f6d3b6825da470 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470 2023-02-13 16:56:10,493 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 16:56:10,494 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1941e9caf41b5697594d4b5192c066a3 in 36ms, sequenceid=45, compaction requested=true 2023-02-13 16:56:10,494 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1941e9caf41b5697594d4b5192c066a3: 2023-02-13 16:56:10,494 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,494 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:10,494 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:10,494 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,494 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,495 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,496 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,496 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,496 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,496 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 16:56:10,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@304ccda0[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:10,498 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 16:56:10,498 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,499 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,499 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,500 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,500 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,500 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,500 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:10,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,500 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,500 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,500 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,500 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,500 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,500 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:10,500 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:10,500 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,506 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:56:10,511 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 16:56:10,511 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 16:56:10,511 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:10,511 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:10,511 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 16:56:10,511 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:10,513 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,513 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,513 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,513 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,513 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,513 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,514 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,514 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,515 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,515 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,517 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,517 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,517 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,526 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,526 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 16:56:10,527 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:10,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,526 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,527 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,527 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,527 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,527 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,527 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,527 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,527 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,527 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:10,527 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:10,627 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:10,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:10,638 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:10,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:10,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 16:56:10,640 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 16:56:10,640 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,640 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 16:56:10,640 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 16:56:10,640 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,640 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,641 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,641 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,642 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,641 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,642 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,642 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,642 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,642 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,642 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,643 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,643 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,643 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,643 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,643 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,643 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,643 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,643 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,644 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,643 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,644 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,644 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,644 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,644 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,644 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,644 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:10,644 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,644 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:10,644 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,644 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. started... 2023-02-13 16:56:10,644 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1941e9caf41b5697594d4b5192c066a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:10,645 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,647 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,654 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/4d7cc9e526654c708bc668a2cef341c0 2023-02-13 16:56:10,664 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/d34522bbb8aa44499add4caf9f635123 2023-02-13 16:56:10,667 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/4d7cc9e526654c708bc668a2cef341c0 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0 2023-02-13 16:56:10,670 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:56:10,671 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/d34522bbb8aa44499add4caf9f635123 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123 2023-02-13 16:56:10,673 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 16:56:10,674 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1941e9caf41b5697594d4b5192c066a3 in 30ms, sequenceid=58, compaction requested=true 2023-02-13 16:56:10,674 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1941e9caf41b5697594d4b5192c066a3: 2023-02-13 16:56:10,674 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,674 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:10,674 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:10,674 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,674 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,674 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,676 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,676 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,676 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,678 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 16:56:10,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53928d1f[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:10,680 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 16:56:10,681 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,682 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,682 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,682 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,682 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:10,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,682 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:10,682 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,682 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,682 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,682 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:10,682 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:10,684 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,684 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,684 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,684 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,684 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,684 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,684 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,685 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,685 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,685 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:56:10,689 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 16:56:10,689 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 16:56:10,689 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:10,689 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:10,689 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 16:56:10,689 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:10,690 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,690 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,691 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,691 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,691 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,690 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,690 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,691 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,692 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,692 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,694 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,694 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,694 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,697 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,697 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,697 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,697 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,697 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,697 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,697 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,697 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,697 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 16:56:10,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,697 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,698 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:10,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,697 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,698 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,704 ERROR [BP-2104803542-172.31.14.131-1676307258605 heartbeating to localhost/127.0.0.1:33763] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104803542-172.31.14.131-1676307258605 (Datanode Uuid 1fadd7f6-283b-4c52-81b3-6a5231ac6c2e) service to localhost/127.0.0.1:33763 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:56:10,798 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:10,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:10,812 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:10,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:10,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 16:56:10,814 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 16:56:10,814 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,814 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 16:56:10,814 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 16:56:10,814 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,814 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,816 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,816 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,816 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,817 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:10,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 16:56:10,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 16:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 16:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 16:56:10,819 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,819 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41587,1676307262998' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,819 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,819 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,819 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34717,1676307263076' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,819 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 16:56:10,819 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:10,819 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:10,819 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,819 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. started... 2023-02-13 16:56:10,819 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1941e9caf41b5697594d4b5192c066a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 16:56:10,821 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,822 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,822 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,822 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,822 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,822 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,834 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/ecb57d2a8cf440068b824dbdec00bef3 2023-02-13 16:56:10,854 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/dc44528e746d4563bfd6fa54766c6922 2023-02-13 16:56:10,857 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/ecb57d2a8cf440068b824dbdec00bef3 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3 2023-02-13 16:56:10,861 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:56:10,861 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/dc44528e746d4563bfd6fa54766c6922 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922 2023-02-13 16:56:10,865 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 16:56:10,866 INFO [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1941e9caf41b5697594d4b5192c066a3 in 47ms, sequenceid=71, compaction requested=true 2023-02-13 16:56:10,866 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1941e9caf41b5697594d4b5192c066a3: 2023-02-13 16:56:10,866 DEBUG [rs(jenkins-hbase4.apache.org,38051,1676307263041)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:10,866 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:10,866 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:10,866 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,866 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 16:56:10,866 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 16:56:10,867 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,867 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,868 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,868 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:10,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38051,1676307263041' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 16:56:10,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e57b00[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:10,870 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 16:56:10,870 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,871 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,871 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,871 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,871 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 16:56:10,872 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,872 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41587,1676307262998' in zk 2023-02-13 16:56:10,871 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,872 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:10,871 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38051,1676307263041' in zk 2023-02-13 16:56:10,871 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 16:56:10,872 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34717,1676307263076' in zk 2023-02-13 16:56:10,877 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,877 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,877 DEBUG [member: 'jenkins-hbase4.apache.org,41587,1676307262998' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,878 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,878 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,878 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,878 DEBUG [member: 'jenkins-hbase4.apache.org,38051,1676307263041' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,879 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 16:56:10,879 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,879 DEBUG [member: 'jenkins-hbase4.apache.org,34717,1676307263076' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 16:56:10,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,38051,1676307263041': 2023-02-13 16:56:10,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38051,1676307263041' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 16:56:10,882 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 16:56:10,882 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:10,882 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:10,882 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 16:56:10,882 INFO [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:10,884 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,884 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,884 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,884 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,884 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,884 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:10,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:10,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,884 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,886 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:10,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,888 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,888 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:10,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:10,890 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 16:56:10,890 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,891 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,893 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,893 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,894 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,894 DEBUG [(jenkins-hbase4.apache.org,45273,1676307261815)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:10,894 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,893 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,893 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:10,894 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:10,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:10,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,894 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 16:56:10,894 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:10,894 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,894 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,894 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 16:56:10,894 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:10,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,894 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:10,895 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 16:56:10,894 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,895 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:10,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:10,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:10,895 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:10,895 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:10,895 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 16:56:10,895 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 16:56:10,995 DEBUG [Listener at localhost/38547] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:10,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:11,002 DEBUG [Listener at localhost/38547] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:11,003 DEBUG [Listener at localhost/38547] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=1941e9caf41b5697594d4b5192c066a3, daughterA=e2df52b565a9a9bfde5865dc84b56ea4, daughterB=5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,004 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=1941e9caf41b5697594d4b5192c066a3, daughterA=e2df52b565a9a9bfde5865dc84b56ea4, daughterB=5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,004 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=1941e9caf41b5697594d4b5192c066a3, daughterA=e2df52b565a9a9bfde5865dc84b56ea4, daughterB=5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,004 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=1941e9caf41b5697594d4b5192c066a3, daughterA=e2df52b565a9a9bfde5865dc84b56ea4, daughterB=5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=1941e9caf41b5697594d4b5192c066a3, UNASSIGN}] 2023-02-13 16:56:11,006 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=1941e9caf41b5697594d4b5192c066a3, UNASSIGN 2023-02-13 16:56:11,007 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=1941e9caf41b5697594d4b5192c066a3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:11,007 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307371007"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307371007"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307371007"}]},"ts":"1676307371007"} 2023-02-13 16:56:11,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 1941e9caf41b5697594d4b5192c066a3, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:56:11,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1941e9caf41b5697594d4b5192c066a3, disabling compactions & flushes 2023-02-13 16:56:11,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:11,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:11,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. after waiting 0 ms 2023-02-13 16:56:11,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:11,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1941e9caf41b5697594d4b5192c066a3 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 16:56:11,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/4e61780ca82b44abbb9813693ca3b4bc 2023-02-13 16:56:11,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/12e66e0f5fbe4a13bb18a3f356e24905 2023-02-13 16:56:11,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf1/4e61780ca82b44abbb9813693ca3b4bc as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc 2023-02-13 16:56:11,196 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:56:11,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/.tmp/cf2/12e66e0f5fbe4a13bb18a3f356e24905 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905 2023-02-13 16:56:11,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 16:56:11,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1941e9caf41b5697594d4b5192c066a3 in 41ms, sequenceid=82, compaction requested=true 2023-02-13 16:56:11,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 16:56:11,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:11,208 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. 2023-02-13 16:56:11,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1941e9caf41b5697594d4b5192c066a3: Waiting for close lock at 1676307371160Running coprocessor pre-close hooks at 1676307371160Disabling compacts and flushes for region at 1676307371160Disabling writes for close at 1676307371160Obtaining lock to block concurrent updates at 1676307371160Preparing flush snapshotting stores in 1941e9caf41b5697594d4b5192c066a3 at 1676307371160Finished memstore snapshotting testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676307371160Flushing stores of testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. at 1676307371161 (+1 ms)Flushing 1941e9caf41b5697594d4b5192c066a3/cf1: creating writer at 1676307371162 (+1 ms)Flushing 1941e9caf41b5697594d4b5192c066a3/cf1: appending metadata at 1676307371164 (+2 ms)Flushing 1941e9caf41b5697594d4b5192c066a3/cf1: closing flushed file at 1676307371164Flushing 1941e9caf41b5697594d4b5192c066a3/cf2: creating writer at 1676307371181 (+17 ms)Flushing 1941e9caf41b5697594d4b5192c066a3/cf2: appending metadata at 1676307371183 (+2 ms)Flushing 1941e9caf41b5697594d4b5192c066a3/cf2: closing flushed file at 1676307371183Flushing 1941e9caf41b5697594d4b5192c066a3/cf1: reopening flushed file at 1676307371193 (+10 ms)Flushing 1941e9caf41b5697594d4b5192c066a3/cf2: reopening flushed file at 1676307371197 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1941e9caf41b5697594d4b5192c066a3 in 41ms, sequenceid=82, compaction requested=true at 1676307371201 (+4 ms)Writing region close event to WAL at 1676307371204 (+3 ms)Running coprocessor post-close hooks at 1676307371208 (+4 ms)Closed at 1676307371208 2023-02-13 16:56:11,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,210 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=1941e9caf41b5697594d4b5192c066a3, regionState=CLOSED 2023-02-13 16:56:11,210 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307371210"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307371210"}]},"ts":"1676307371210"} 2023-02-13 16:56:11,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-13 16:56:11,213 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 1941e9caf41b5697594d4b5192c066a3, server=jenkins-hbase4.apache.org,38051,1676307263041 in 203 msec 2023-02-13 16:56:11,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-13 16:56:11,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=1941e9caf41b5697594d4b5192c066a3, UNASSIGN in 207 msec 2023-02-13 16:56:11,225 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=1941e9caf41b5697594d4b5192c066a3, threads=14 2023-02-13 16:56:11,225 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,225 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,225 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,225 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,225 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,225 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,226 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,226 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,226 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,226 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,227 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,227 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,227 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,227 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,448 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,452 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,456 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,508 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,528 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,531 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,532 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,532 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,537 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,544 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,552 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,552 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,559 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,560 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905 for region: 1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:11,560 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 1941e9caf41b5697594d4b5192c066a3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 16:56:11,618 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:56:11,620 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 16:56:11,621 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676307371621"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676307371621"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676307371621"}]},"ts":"1676307371621"} 2023-02-13 16:56:11,621 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307371621"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307371621"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307371621"}]},"ts":"1676307371621"} 2023-02-13 16:56:11,621 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307371621"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307371621"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307371621"}]},"ts":"1676307371621"} 2023-02-13 16:56:11,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e2df52b565a9a9bfde5865dc84b56ea4, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=5cff619736fb952f3797c50c6dd76513, ASSIGN}] 2023-02-13 16:56:11,625 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=5cff619736fb952f3797c50c6dd76513, ASSIGN 2023-02-13 16:56:11,625 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=5cff619736fb952f3797c50c6dd76513, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38051,1676307263041; forceNewPlan=false, retain=false 2023-02-13 16:56:11,626 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e2df52b565a9a9bfde5865dc84b56ea4, ASSIGN 2023-02-13 16:56:11,626 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=e2df52b565a9a9bfde5865dc84b56ea4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38051,1676307263041; forceNewPlan=false, retain=false 2023-02-13 16:56:11,776 INFO [jenkins-hbase4:45273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 16:56:11,777 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=5cff619736fb952f3797c50c6dd76513, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:11,777 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307371777"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307371777"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307371777"}]},"ts":"1676307371777"} 2023-02-13 16:56:11,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 5cff619736fb952f3797c50c6dd76513, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:56:11,780 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=e2df52b565a9a9bfde5865dc84b56ea4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:11,780 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307371780"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307371780"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307371780"}]},"ts":"1676307371780"} 2023-02-13 16:56:11,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure e2df52b565a9a9bfde5865dc84b56ea4, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:56:11,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:11,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5cff619736fb952f3797c50c6dd76513, NAME => 'testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 16:56:11,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:56:11,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:11,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,933 INFO [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,934 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1 2023-02-13 16:56:11,934 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1 2023-02-13 16:56:11,934 INFO [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5cff619736fb952f3797c50c6dd76513 columnFamilyName cf1 2023-02-13 16:56:11,946 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0-top 2023-02-13 16:56:11,949 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc-top 2023-02-13 16:56:11,952 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4-top 2023-02-13 16:56:11,955 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302-top 2023-02-13 16:56:11,958 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e-top 2023-02-13 16:56:11,961 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255-top 2023-02-13 16:56:11,964 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3-top 2023-02-13 16:56:11,964 INFO [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(310): Store=5cff619736fb952f3797c50c6dd76513/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:11,964 INFO [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,965 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2 2023-02-13 16:56:11,965 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2 2023-02-13 16:56:11,965 INFO [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5cff619736fb952f3797c50c6dd76513 columnFamilyName cf2 2023-02-13 16:56:11,976 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905-top 2023-02-13 16:56:11,979 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90-top 2023-02-13 16:56:11,983 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6-top 2023-02-13 16:56:11,986 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123-top 2023-02-13 16:56:11,989 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922-top 2023-02-13 16:56:11,992 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470-top 2023-02-13 16:56:11,995 DEBUG [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92-top 2023-02-13 16:56:11,995 INFO [StoreOpener-5cff619736fb952f3797c50c6dd76513-1] regionserver.HStore(310): Store=5cff619736fb952f3797c50c6dd76513/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:11,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:11,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:11,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:12,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5cff619736fb952f3797c50c6dd76513; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59084119, jitterRate=-0.11957801878452301}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:12,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5cff619736fb952f3797c50c6dd76513: Running coprocessor pre-open hook at 1676307371932Writing region info on filesystem at 1676307371932Initializing all the Stores at 1676307371933 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307371933Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307371933Cleaning up temporary data from old regions at 1676307371998 (+65 ms)Cleaning up detritus from prior splits at 1676307371998Running coprocessor post-open hooks at 1676307372000 (+2 ms)Region opened successfully at 1676307372000 2023-02-13 16:56:12,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513., pid=149, masterSystemTime=1676307371929 2023-02-13 16:56:12,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:12,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:56:12,002 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:12,002 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:12,002 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,002 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,002 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1912): 5cff619736fb952f3797c50c6dd76513/cf1 is initiating minor compaction (all files) 2023-02-13 16:56:12,002 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1912): 5cff619736fb952f3797c50c6dd76513/cf2 is initiating minor compaction (all files) 2023-02-13 16:56:12,002 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5cff619736fb952f3797c50c6dd76513/cf1 in testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,002 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5cff619736fb952f3797c50c6dd76513/cf2 in testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,002 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc-top] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/.tmp, totalSize=34.9 K 2023-02-13 16:56:12,002 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905-top] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/.tmp, totalSize=34.9 K 2023-02-13 16:56:12,003 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307369906 2023-02-13 16:56:12,003 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676307369906 2023-02-13 16:56:12,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,003 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307370074 2023-02-13 16:56:12,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e2df52b565a9a9bfde5865dc84b56ea4, NAME => 'testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 16:56:12,003 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=5cff619736fb952f3797c50c6dd76513, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:12,003 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676307370074 2023-02-13 16:56:12,004 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307372003"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307372003"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307372003"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307372003"}]},"ts":"1676307372003"} 2023-02-13 16:56:12,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 16:56:12,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:12,004 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307370256 2023-02-13 16:56:12,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,004 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676307370256 2023-02-13 16:56:12,004 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676307370442 2023-02-13 16:56:12,005 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676307370442 2023-02-13 16:56:12,005 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676307370629 2023-02-13 16:56:12,005 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676307370800 2023-02-13 16:56:12,005 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676307370629 2023-02-13 16:56:12,005 INFO [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,006 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting 12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307370996 2023-02-13 16:56:12,006 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676307370800 2023-02-13 16:56:12,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-02-13 16:56:12,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 5cff619736fb952f3797c50c6dd76513, server=jenkins-hbase4.apache.org,38051,1676307263041 in 227 msec 2023-02-13 16:56:12,007 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676307370996 2023-02-13 16:56:12,008 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1 2023-02-13 16:56:12,008 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1 2023-02-13 16:56:12,008 INFO [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e2df52b565a9a9bfde5865dc84b56ea4 columnFamilyName cf1 2023-02-13 16:56:12,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=5cff619736fb952f3797c50c6dd76513, ASSIGN in 383 msec 2023-02-13 16:56:12,020 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] throttle.PressureAwareThroughputController(145): 5cff619736fb952f3797c50c6dd76513#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:12,021 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5cff619736fb952f3797c50c6dd76513#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:12,022 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0-bottom 2023-02-13 16:56:12,034 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc-bottom 2023-02-13 16:56:12,038 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4-bottom 2023-02-13 16:56:12,041 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302-bottom 2023-02-13 16:56:12,049 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e-bottom 2023-02-13 16:56:12,049 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/.tmp/cf2/90a6ebdac4f84a9885eb40e91d65efa6 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/90a6ebdac4f84a9885eb40e91d65efa6 2023-02-13 16:56:12,053 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255-bottom 2023-02-13 16:56:12,055 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5cff619736fb952f3797c50c6dd76513/cf2 of 5cff619736fb952f3797c50c6dd76513 into 90a6ebdac4f84a9885eb40e91d65efa6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:12,055 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5cff619736fb952f3797c50c6dd76513: 2023-02-13 16:56:12,055 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513., storeName=5cff619736fb952f3797c50c6dd76513/cf2, priority=9, startTime=1676307372002; duration=0sec 2023-02-13 16:56:12,055 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:12,055 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/.tmp/cf1/faabce5b17a94cbb9a2f76930472e090 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/faabce5b17a94cbb9a2f76930472e090 2023-02-13 16:56:12,057 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3-bottom 2023-02-13 16:56:12,057 INFO [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(310): Store=e2df52b565a9a9bfde5865dc84b56ea4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:12,057 INFO [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,058 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2 2023-02-13 16:56:12,058 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2 2023-02-13 16:56:12,058 INFO [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e2df52b565a9a9bfde5865dc84b56ea4 columnFamilyName cf2 2023-02-13 16:56:12,060 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5cff619736fb952f3797c50c6dd76513/cf1 of 5cff619736fb952f3797c50c6dd76513 into faabce5b17a94cbb9a2f76930472e090(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:12,060 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5cff619736fb952f3797c50c6dd76513: 2023-02-13 16:56:12,060 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513., storeName=5cff619736fb952f3797c50c6dd76513/cf1, priority=9, startTime=1676307372001; duration=0sec 2023-02-13 16:56:12,060 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:12,070 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905-bottom 2023-02-13 16:56:12,072 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90-bottom 2023-02-13 16:56:12,075 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6-bottom 2023-02-13 16:56:12,078 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123-bottom 2023-02-13 16:56:12,081 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922-bottom 2023-02-13 16:56:12,084 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470-bottom 2023-02-13 16:56:12,089 DEBUG [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(539): loaded hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92-bottom 2023-02-13 16:56:12,090 INFO [StoreOpener-e2df52b565a9a9bfde5865dc84b56ea4-1] regionserver.HStore(310): Store=e2df52b565a9a9bfde5865dc84b56ea4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:12,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:12,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e2df52b565a9a9bfde5865dc84b56ea4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59429847, jitterRate=-0.11442627012729645}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:12,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e2df52b565a9a9bfde5865dc84b56ea4: Running coprocessor pre-open hook at 1676307372004Writing region info on filesystem at 1676307372004Initializing all the Stores at 1676307372005 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307372005Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307372005Cleaning up temporary data from old regions at 1676307372091 (+86 ms)Cleaning up detritus from prior splits at 1676307372092 (+1 ms)Running coprocessor post-open hooks at 1676307372094 (+2 ms)Region opened successfully at 1676307372094 2023-02-13 16:56:12,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4., pid=150, masterSystemTime=1676307371929 2023-02-13 16:56:12,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:12,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:56:12,095 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:12,095 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 16:56:12,096 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,096 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,096 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1912): e2df52b565a9a9bfde5865dc84b56ea4/cf1 is initiating minor compaction (all files) 2023-02-13 16:56:12,096 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1912): e2df52b565a9a9bfde5865dc84b56ea4/cf2 is initiating minor compaction (all files) 2023-02-13 16:56:12,096 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e2df52b565a9a9bfde5865dc84b56ea4/cf1 in testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,096 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e2df52b565a9a9bfde5865dc84b56ea4/cf2 in testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,096 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/.tmp, totalSize=34.9 K 2023-02-13 16:56:12,096 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905-bottom] into tmpdir=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/.tmp, totalSize=34.9 K 2023-02-13 16:56:12,097 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307369906 2023-02-13 16:56:12,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,097 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676307369906 2023-02-13 16:56:12,097 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307370074 2023-02-13 16:56:12,097 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=e2df52b565a9a9bfde5865dc84b56ea4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:12,097 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676307370074 2023-02-13 16:56:12,097 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307372097"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307372097"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307372097"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307372097"}]},"ts":"1676307372097"} 2023-02-13 16:56:12,098 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307370256 2023-02-13 16:56:12,098 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676307370256 2023-02-13 16:56:12,098 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307370442 2023-02-13 16:56:12,098 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676307370442 2023-02-13 16:56:12,098 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307370629 2023-02-13 16:56:12,098 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676307370629 2023-02-13 16:56:12,099 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307370800 2023-02-13 16:56:12,099 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676307370800 2023-02-13 16:56:12,099 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] compactions.Compactor(207): Compacting 12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307370996 2023-02-13 16:56:12,100 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] compactions.Compactor(207): Compacting 4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676307370996 2023-02-13 16:56:12,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-02-13 16:56:12,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure e2df52b565a9a9bfde5865dc84b56ea4, server=jenkins-hbase4.apache.org,38051,1676307263041 in 317 msec 2023-02-13 16:56:12,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-13 16:56:12,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e2df52b565a9a9bfde5865dc84b56ea4, ASSIGN in 477 msec 2023-02-13 16:56:12,104 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=1941e9caf41b5697594d4b5192c066a3, daughterA=e2df52b565a9a9bfde5865dc84b56ea4, daughterB=5cff619736fb952f3797c50c6dd76513 in 1.1010 sec 2023-02-13 16:56:12,111 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] throttle.PressureAwareThroughputController(145): e2df52b565a9a9bfde5865dc84b56ea4#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:12,111 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] throttle.PressureAwareThroughputController(145): e2df52b565a9a9bfde5865dc84b56ea4#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:12,130 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/.tmp/cf1/88747c89f68144c595a3a676b02e5f6b as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/88747c89f68144c595a3a676b02e5f6b 2023-02-13 16:56:12,134 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e2df52b565a9a9bfde5865dc84b56ea4/cf1 of e2df52b565a9a9bfde5865dc84b56ea4 into 88747c89f68144c595a3a676b02e5f6b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:12,135 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e2df52b565a9a9bfde5865dc84b56ea4: 2023-02-13 16:56:12,135 INFO [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4., storeName=e2df52b565a9a9bfde5865dc84b56ea4/cf1, priority=9, startTime=1676307372094; duration=0sec 2023-02-13 16:56:12,135 DEBUG [RS:1;jenkins-hbase4:38051-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:12,143 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/.tmp/cf2/abf1d3458b9c4c64bd1dca1dcfce014e as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/abf1d3458b9c4c64bd1dca1dcfce014e 2023-02-13 16:56:12,147 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e2df52b565a9a9bfde5865dc84b56ea4/cf2 of e2df52b565a9a9bfde5865dc84b56ea4 into abf1d3458b9c4c64bd1dca1dcfce014e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:12,148 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e2df52b565a9a9bfde5865dc84b56ea4: 2023-02-13 16:56:12,148 INFO [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4., storeName=e2df52b565a9a9bfde5865dc84b56ea4/cf2, priority=9, startTime=1676307372095; duration=0sec 2023-02-13 16:56:12,148 DEBUG [RS:1;jenkins-hbase4:38051-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:12,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38051] ipc.CallRunner(144): callId: 1156 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:45068 deadline: 1676307432254, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3. is not online on jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:12,368 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 16:56:12,380 INFO [Listener at localhost/38547] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 16:56:12,385 DEBUG [Listener at localhost/38547] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 16:56:12,398 WARN [Listener at localhost/38547] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 16:56:12,400 INFO [Listener at localhost/38547] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-13 16:56:12,401 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-13 16:56:12,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-13 16:56:12,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 16:56:12,404 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307372404"}]},"ts":"1676307372404"} 2023-02-13 16:56:12,405 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-13 16:56:12,407 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-13 16:56:12,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e2df52b565a9a9bfde5865dc84b56ea4, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=5cff619736fb952f3797c50c6dd76513, UNASSIGN}] 2023-02-13 16:56:12,408 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=5cff619736fb952f3797c50c6dd76513, UNASSIGN 2023-02-13 16:56:12,409 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=5cff619736fb952f3797c50c6dd76513, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:12,409 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307372409"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307372409"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307372409"}]},"ts":"1676307372409"} 2023-02-13 16:56:12,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 5cff619736fb952f3797c50c6dd76513, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:56:12,412 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e2df52b565a9a9bfde5865dc84b56ea4, UNASSIGN 2023-02-13 16:56:12,412 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=e2df52b565a9a9bfde5865dc84b56ea4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:12,412 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307372412"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307372412"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307372412"}]},"ts":"1676307372412"} 2023-02-13 16:56:12,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure e2df52b565a9a9bfde5865dc84b56ea4, server=jenkins-hbase4.apache.org,38051,1676307263041}] 2023-02-13 16:56:12,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 16:56:12,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e2df52b565a9a9bfde5865dc84b56ea4, disabling compactions & flushes 2023-02-13 16:56:12,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. after waiting 0 ms 2023-02-13 16:56:12,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,563 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc-bottom] to archive 2023-02-13 16:56:12,564 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:12,565 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,567 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,568 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,569 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,571 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,572 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,573 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,575 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922-bottom, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905-bottom] to archive 2023-02-13 16:56:12,575 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:12,577 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,578 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,579 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,581 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,583 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,584 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,585 DEBUG [StoreCloser-testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:56:12,589 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:12,589 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4. 2023-02-13 16:56:12,589 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e2df52b565a9a9bfde5865dc84b56ea4: Waiting for close lock at 1676307372562Running coprocessor pre-close hooks at 1676307372562Disabling compacts and flushes for region at 1676307372562Disabling writes for close at 1676307372562Writing region close event to WAL at 1676307372586 (+24 ms)Running coprocessor post-close hooks at 1676307372589 (+3 ms)Closed at 1676307372589 2023-02-13 16:56:12,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:12,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5cff619736fb952f3797c50c6dd76513, disabling compactions & flushes 2023-02-13 16:56:12,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. after waiting 0 ms 2023-02-13 16:56:12,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,591 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=e2df52b565a9a9bfde5865dc84b56ea4, regionState=CLOSED 2023-02-13 16:56:12,591 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307372591"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307372591"}]},"ts":"1676307372591"} 2023-02-13 16:56:12,592 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc-top] to archive 2023-02-13 16:56:12,592 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:12,594 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-13 16:56:12,594 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure e2df52b565a9a9bfde5865dc84b56ea4, server=jenkins-hbase4.apache.org,38051,1676307263041 in 180 msec 2023-02-13 16:56:12,595 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/535e99663dd142b892e5f90f11f95da4.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e2df52b565a9a9bfde5865dc84b56ea4, UNASSIGN in 187 msec 2023-02-13 16:56:12,596 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/73582f3711d043a2b105067b15a3c302.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,597 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/97320f225b0a4b14a5dfdeb01b068255.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,598 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/800bb38182e44e1ba943f82908fc677e.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,599 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4d7cc9e526654c708bc668a2cef341c0.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,600 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/ecb57d2a8cf440068b824dbdec00bef3.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,601 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/4e61780ca82b44abbb9813693ca3b4bc.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,603 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922-top, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3->hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905-top] to archive 2023-02-13 16:56:12,603 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:12,604 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c2dfe3394d6844b686516bd0ec17ba90.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,605 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/f54475d5c17e49ae93434e756aebed92.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,606 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,607 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/efe858801fd54c2286f6d3b6825da470.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,608 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/d34522bbb8aa44499add4caf9f635123.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,609 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/dc44528e746d4563bfd6fa54766c6922.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,610 DEBUG [StoreCloser-testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/12e66e0f5fbe4a13bb18a3f356e24905.1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 16:56:12,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:12,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513. 2023-02-13 16:56:12,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5cff619736fb952f3797c50c6dd76513: Waiting for close lock at 1676307372590Running coprocessor pre-close hooks at 1676307372590Disabling compacts and flushes for region at 1676307372590Disabling writes for close at 1676307372590Writing region close event to WAL at 1676307372611 (+21 ms)Running coprocessor post-close hooks at 1676307372614 (+3 ms)Closed at 1676307372614 2023-02-13 16:56:12,615 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:12,616 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=5cff619736fb952f3797c50c6dd76513, regionState=CLOSED 2023-02-13 16:56:12,616 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307372616"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307372616"}]},"ts":"1676307372616"} 2023-02-13 16:56:12,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-13 16:56:12,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 5cff619736fb952f3797c50c6dd76513, server=jenkins-hbase4.apache.org,38051,1676307263041 in 207 msec 2023-02-13 16:56:12,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-13 16:56:12,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=5cff619736fb952f3797c50c6dd76513, UNASSIGN in 211 msec 2023-02-13 16:56:12,620 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307372620"}]},"ts":"1676307372620"} 2023-02-13 16:56:12,621 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-13 16:56:12,622 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-13 16:56:12,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 222 msec 2023-02-13 16:56:12,695 ERROR [BP-2104803542-172.31.14.131-1676307258605 heartbeating to localhost/127.0.0.1:33763] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104803542-172.31.14.131-1676307258605 (Datanode Uuid a3585e58-faa4-4a64-84b5-04c6ddf75562) service to localhost/127.0.0.1:33763 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:56:12,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 16:56:12,706 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-13 16:56:12,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-13 16:56:12,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 16:56:12,709 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 16:56:12,710 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 16:56:12,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 16:56:12,714 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,714 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:12,714 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,715 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/recovered.edits] 2023-02-13 16:56:12,715 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/recovered.edits] 2023-02-13 16:56:12,715 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2, FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/recovered.edits] 2023-02-13 16:56:12,718 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/88747c89f68144c595a3a676b02e5f6b to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf1/88747c89f68144c595a3a676b02e5f6b 2023-02-13 16:56:12,719 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/faabce5b17a94cbb9a2f76930472e090 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf1/faabce5b17a94cbb9a2f76930472e090 2023-02-13 16:56:12,719 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4d7cc9e526654c708bc668a2cef341c0 2023-02-13 16:56:12,721 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/4e61780ca82b44abbb9813693ca3b4bc 2023-02-13 16:56:12,721 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/90a6ebdac4f84a9885eb40e91d65efa6 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/cf2/90a6ebdac4f84a9885eb40e91d65efa6 2023-02-13 16:56:12,722 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/abf1d3458b9c4c64bd1dca1dcfce014e to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/cf2/abf1d3458b9c4c64bd1dca1dcfce014e 2023-02-13 16:56:12,723 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/535e99663dd142b892e5f90f11f95da4 2023-02-13 16:56:12,724 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/73582f3711d043a2b105067b15a3c302 2023-02-13 16:56:12,725 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513/recovered.edits/90.seqid 2023-02-13 16:56:12,725 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/recovered.edits/90.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4/recovered.edits/90.seqid 2023-02-13 16:56:12,725 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/800bb38182e44e1ba943f82908fc677e 2023-02-13 16:56:12,726 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/5cff619736fb952f3797c50c6dd76513 2023-02-13 16:56:12,726 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/e2df52b565a9a9bfde5865dc84b56ea4 2023-02-13 16:56:12,727 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/97320f225b0a4b14a5dfdeb01b068255 2023-02-13 16:56:12,728 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf1/ecb57d2a8cf440068b824dbdec00bef3 2023-02-13 16:56:12,730 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/12e66e0f5fbe4a13bb18a3f356e24905 2023-02-13 16:56:12,731 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c2dfe3394d6844b686516bd0ec17ba90 2023-02-13 16:56:12,732 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/c732f5ab5fe14ceea6ae4a824ba9e3f6 2023-02-13 16:56:12,733 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/d34522bbb8aa44499add4caf9f635123 2023-02-13 16:56:12,734 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/dc44528e746d4563bfd6fa54766c6922 2023-02-13 16:56:12,735 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/efe858801fd54c2286f6d3b6825da470 2023-02-13 16:56:12,736 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92 to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/cf2/f54475d5c17e49ae93434e756aebed92 2023-02-13 16:56:12,739 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/recovered.edits/85.seqid to hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/archive/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3/recovered.edits/85.seqid 2023-02-13 16:56:12,739 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/.tmp/data/default/testSplitTableRegion/1941e9caf41b5697594d4b5192c066a3 2023-02-13 16:56:12,739 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 16:56:12,741 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 16:56:12,743 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-13 16:56:12,744 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-13 16:56:12,745 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 16:56:12,745 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-13 16:56:12,746 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307372745"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:12,746 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307372745"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:12,746 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307372745"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:12,747 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 16:56:12,747 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1941e9caf41b5697594d4b5192c066a3, NAME => 'testSplitTableRegion,,1676307369402.1941e9caf41b5697594d4b5192c066a3.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e2df52b565a9a9bfde5865dc84b56ea4, NAME => 'testSplitTableRegion,,1676307371002.e2df52b565a9a9bfde5865dc84b56ea4.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 5cff619736fb952f3797c50c6dd76513, NAME => 'testSplitTableRegion,41,1676307371002.5cff619736fb952f3797c50c6dd76513.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 16:56:12,747 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-13 16:56:12,747 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307372747"}]},"ts":"9223372036854775807"} 2023-02-13 16:56:12,749 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-13 16:56:12,750 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 16:56:12,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 44 msec 2023-02-13 16:56:12,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 16:56:12,812 INFO [Listener at localhost/38547] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-13 16:56:12,828 INFO [Listener at localhost/38547] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=378 (was 375) - Thread LEAK? -, OpenFileDescriptor=834 (was 832) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=107 (was 99) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=1025 (was 1037) 2023-02-13 16:56:12,828 INFO [Listener at localhost/38547] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 16:56:12,828 INFO [Listener at localhost/38547] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:56:12,832 DEBUG [Listener at localhost/38547] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c8d5aec to 127.0.0.1:57428 2023-02-13 16:56:12,833 DEBUG [Listener at localhost/38547] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:12,833 DEBUG [Listener at localhost/38547] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 16:56:12,834 DEBUG [Listener at localhost/38547] util.JVMClusterUtil(257): Found active master hash=1019338664, stopped=false 2023-02-13 16:56:12,834 INFO [Listener at localhost/38547] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:56:12,836 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 16:56:12,836 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 16:56:12,836 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 16:56:12,836 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:56:12,836 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 16:56:12,836 INFO [Listener at localhost/38547] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:12,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:56:12,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:56:12,837 DEBUG [Listener at localhost/38547] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x25e220f6 to 127.0.0.1:57428 2023-02-13 16:56:12,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:56:12,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:56:12,837 DEBUG [Listener at localhost/38547] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:12,837 INFO [Listener at localhost/38547] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,41587,1676307262998' ***** 2023-02-13 16:56:12,838 INFO [Listener at localhost/38547] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 16:56:12,838 INFO [Listener at localhost/38547] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38051,1676307263041' ***** 2023-02-13 16:56:12,838 INFO [Listener at localhost/38547] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 16:56:12,838 INFO [Listener at localhost/38547] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,34717,1676307263076' ***** 2023-02-13 16:56:12,838 INFO [Listener at localhost/38547] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 16:56:12,838 INFO [RS:1;jenkins-hbase4:38051] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 16:56:12,838 INFO [RS:2;jenkins-hbase4:34717] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 16:56:12,838 INFO [RS:0;jenkins-hbase4:41587] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 16:56:12,838 INFO [RS:0;jenkins-hbase4:41587] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 16:56:12,838 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 16:56:12,838 INFO [RS:0;jenkins-hbase4:41587] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 16:56:12,838 INFO [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:12,839 DEBUG [RS:0;jenkins-hbase4:41587] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6db8f005 to 127.0.0.1:57428 2023-02-13 16:56:12,839 DEBUG [RS:0;jenkins-hbase4:41587] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:12,839 INFO [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,41587,1676307262998; all regions closed. 2023-02-13 16:56:12,839 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 16:56:12,839 INFO [RS:1;jenkins-hbase4:38051] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 16:56:12,839 INFO [RS:2;jenkins-hbase4:34717] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 16:56:12,839 INFO [RS:2;jenkins-hbase4:34717] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 16:56:12,839 INFO [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:12,839 INFO [RS:1;jenkins-hbase4:38051] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 16:56:12,839 DEBUG [RS:2;jenkins-hbase4:34717] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e28de38 to 127.0.0.1:57428 2023-02-13 16:56:12,839 INFO [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(3304): Received CLOSE for 49a9767f24746f442cea05a1662ed767 2023-02-13 16:56:12,839 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 16:56:12,839 DEBUG [RS:2;jenkins-hbase4:34717] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:12,840 INFO [RS:2;jenkins-hbase4:34717] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 16:56:12,840 INFO [RS:2;jenkins-hbase4:34717] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 16:56:12,840 INFO [RS:2;jenkins-hbase4:34717] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 16:56:12,840 INFO [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 16:56:12,840 INFO [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:12,840 DEBUG [RS:1;jenkins-hbase4:38051] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5a6d36a2 to 127.0.0.1:57428 2023-02-13 16:56:12,840 DEBUG [RS:1;jenkins-hbase4:38051] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:12,840 INFO [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 16:56:12,840 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1479): Online Regions={49a9767f24746f442cea05a1662ed767=hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767.} 2023-02-13 16:56:12,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 49a9767f24746f442cea05a1662ed767, disabling compactions & flushes 2023-02-13 16:56:12,840 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:56:12,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:56:12,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. after waiting 0 ms 2023-02-13 16:56:12,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:56:12,840 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 49a9767f24746f442cea05a1662ed767 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 16:56:12,841 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1505): Waiting on 49a9767f24746f442cea05a1662ed767 2023-02-13 16:56:12,844 INFO [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 16:56:12,844 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-13 16:56:12,844 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 16:56:12,845 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 16:56:12,845 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 16:56:12,845 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 16:56:12,845 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 16:56:12,845 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 16:56:12,845 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-13 16:56:13,038 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 16:56:13,038 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 16:56:13,038 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 16:56:13,041 DEBUG [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1505): Waiting on 49a9767f24746f442cea05a1662ed767 2023-02-13 16:56:13,045 DEBUG [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 16:56:13,053 DEBUG [RS:0;jenkins-hbase4:41587] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/oldWALs 2023-02-13 16:56:13,054 INFO [RS:0;jenkins-hbase4:41587] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41587%2C1676307262998:(num 1676307264541) 2023-02-13 16:56:13,054 DEBUG [RS:0;jenkins-hbase4:41587] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:13,054 INFO [RS:0;jenkins-hbase4:41587] regionserver.LeaseManager(133): Closed leases 2023-02-13 16:56:13,056 INFO [RS:0;jenkins-hbase4:41587] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 16:56:13,056 INFO [RS:0;jenkins-hbase4:41587] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 16:56:13,056 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 16:56:13,056 INFO [RS:0;jenkins-hbase4:41587] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 16:56:13,057 INFO [RS:0;jenkins-hbase4:41587] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 16:56:13,061 INFO [RS:0;jenkins-hbase4:41587] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41587 2023-02-13 16:56:13,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767/.tmp/info/97c0c1ded5e84913b92e2cc4342b899d 2023-02-13 16:56:13,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:13,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:13,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41587,1676307262998 2023-02-13 16:56:13,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 16:56:13,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 16:56:13,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 16:56:13,067 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@21c7d6a5 rejected from java.util.concurrent.ThreadPoolExecutor@25f84645[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,067 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 16:56:13,067 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4ebd2c31 rejected from java.util.concurrent.ThreadPoolExecutor@25f84645[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,068 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41587,1676307262998] 2023-02-13 16:56:13,069 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41587,1676307262998; numProcessing=1 2023-02-13 16:56:13,069 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/.tmp/info/437c6722f9f544a7b0a9fbcc6bf6b633 2023-02-13 16:56:13,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767/.tmp/info/97c0c1ded5e84913b92e2cc4342b899d as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767/info/97c0c1ded5e84913b92e2cc4342b899d 2023-02-13 16:56:13,070 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41587,1676307262998 already deleted, retry=false 2023-02-13 16:56:13,070 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41587,1676307262998 expired; onlineServers=2 2023-02-13 16:56:13,072 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 437c6722f9f544a7b0a9fbcc6bf6b633 2023-02-13 16:56:13,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767/info/97c0c1ded5e84913b92e2cc4342b899d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 16:56:13,075 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 49a9767f24746f442cea05a1662ed767 in 235ms, sequenceid=6, compaction requested=false 2023-02-13 16:56:13,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/namespace/49a9767f24746f442cea05a1662ed767/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 16:56:13,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:13,081 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:56:13,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 49a9767f24746f442cea05a1662ed767: Waiting for close lock at 1676307372840Running coprocessor pre-close hooks at 1676307372840Disabling compacts and flushes for region at 1676307372840Disabling writes for close at 1676307372840Obtaining lock to block concurrent updates at 1676307372840Preparing flush snapshotting stores in 49a9767f24746f442cea05a1662ed767 at 1676307372840Finished memstore snapshotting hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676307372841 (+1 ms)Flushing stores of hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. at 1676307372841Flushing 49a9767f24746f442cea05a1662ed767/info: creating writer at 1676307372841Flushing 49a9767f24746f442cea05a1662ed767/info: appending metadata at 1676307372852 (+11 ms)Flushing 49a9767f24746f442cea05a1662ed767/info: closing flushed file at 1676307372852Flushing 49a9767f24746f442cea05a1662ed767/info: reopening flushed file at 1676307373070 (+218 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 49a9767f24746f442cea05a1662ed767 in 235ms, sequenceid=6, compaction requested=false at 1676307373075 (+5 ms)Writing region close event to WAL at 1676307373077 (+2 ms)Running coprocessor post-close hooks at 1676307373080 (+3 ms)Closed at 1676307373081 (+1 ms) 2023-02-13 16:56:13,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676307264948.49a9767f24746f442cea05a1662ed767. 2023-02-13 16:56:13,082 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/.tmp/rep_barrier/63890502fb0548bca9ec1dccf031a603 2023-02-13 16:56:13,085 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63890502fb0548bca9ec1dccf031a603 2023-02-13 16:56:13,096 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/.tmp/table/a834e49c342e42c8b86954c5448e9c16 2023-02-13 16:56:13,100 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a834e49c342e42c8b86954c5448e9c16 2023-02-13 16:56:13,100 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/.tmp/info/437c6722f9f544a7b0a9fbcc6bf6b633 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/info/437c6722f9f544a7b0a9fbcc6bf6b633 2023-02-13 16:56:13,103 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 437c6722f9f544a7b0a9fbcc6bf6b633 2023-02-13 16:56:13,103 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/info/437c6722f9f544a7b0a9fbcc6bf6b633, entries=59, sequenceid=210, filesize=11.1 K 2023-02-13 16:56:13,104 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/.tmp/rep_barrier/63890502fb0548bca9ec1dccf031a603 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/rep_barrier/63890502fb0548bca9ec1dccf031a603 2023-02-13 16:56:13,107 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63890502fb0548bca9ec1dccf031a603 2023-02-13 16:56:13,107 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/rep_barrier/63890502fb0548bca9ec1dccf031a603, entries=24, sequenceid=210, filesize=7.5 K 2023-02-13 16:56:13,107 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/.tmp/table/a834e49c342e42c8b86954c5448e9c16 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/table/a834e49c342e42c8b86954c5448e9c16 2023-02-13 16:56:13,110 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a834e49c342e42c8b86954c5448e9c16 2023-02-13 16:56:13,110 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/table/a834e49c342e42c8b86954c5448e9c16, entries=36, sequenceid=210, filesize=8.0 K 2023-02-13 16:56:13,111 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 266ms, sequenceid=210, compaction requested=false 2023-02-13 16:56:13,115 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-13 16:56:13,115 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 16:56:13,116 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 16:56:13,116 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 16:56:13,116 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676307372844Running coprocessor pre-close hooks at 1676307372844Disabling compacts and flushes for region at 1676307372844Disabling writes for close at 1676307372845 (+1 ms)Obtaining lock to block concurrent updates at 1676307372845Preparing flush snapshotting stores in 1588230740 at 1676307372845Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1676307372845Flushing stores of hbase:meta,,1.1588230740 at 1676307372853 (+8 ms)Flushing 1588230740/info: creating writer at 1676307373038 (+185 ms)Flushing 1588230740/info: appending metadata at 1676307373046 (+8 ms)Flushing 1588230740/info: closing flushed file at 1676307373046Flushing 1588230740/rep_barrier: creating writer at 1676307373073 (+27 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676307373075 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676307373075Flushing 1588230740/table: creating writer at 1676307373085 (+10 ms)Flushing 1588230740/table: appending metadata at 1676307373087 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676307373087Flushing 1588230740/info: reopening flushed file at 1676307373101 (+14 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676307373104 (+3 ms)Flushing 1588230740/table: reopening flushed file at 1676307373108 (+4 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 266ms, sequenceid=210, compaction requested=false at 1676307373111 (+3 ms)Writing region close event to WAL at 1676307373113 (+2 ms)Running coprocessor post-close hooks at 1676307373115 (+2 ms)Closed at 1676307373116 (+1 ms) 2023-02-13 16:56:13,117 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 16:56:13,168 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:13,169 INFO [RS:0;jenkins-hbase4:41587] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,41587,1676307262998; zookeeper connection closed. 2023-02-13 16:56:13,169 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@36fd17bc rejected from java.util.concurrent.ThreadPoolExecutor@25f84645[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,169 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:41587-0x1004e9acff70001, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:13,169 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@272e5b5e rejected from java.util.concurrent.ThreadPoolExecutor@25f84645[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,169 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@795979e2] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@795979e2 2023-02-13 16:56:13,241 INFO [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38051,1676307263041; all regions closed. 2023-02-13 16:56:13,245 INFO [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,34717,1676307263076; all regions closed. 2023-02-13 16:56:13,249 DEBUG [RS:1;jenkins-hbase4:38051] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/oldWALs 2023-02-13 16:56:13,249 INFO [RS:1;jenkins-hbase4:38051] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38051%2C1676307263041:(num 1676307264541) 2023-02-13 16:56:13,249 DEBUG [RS:1;jenkins-hbase4:38051] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:13,250 INFO [RS:1;jenkins-hbase4:38051] regionserver.LeaseManager(133): Closed leases 2023-02-13 16:56:13,251 DEBUG [RS:2;jenkins-hbase4:34717] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/oldWALs 2023-02-13 16:56:13,251 INFO [RS:2;jenkins-hbase4:34717] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34717%2C1676307263076.meta:.meta(num 1676307264685) 2023-02-13 16:56:13,252 INFO [RS:1;jenkins-hbase4:38051] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 16:56:13,252 INFO [RS:1;jenkins-hbase4:38051] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 16:56:13,252 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 16:56:13,252 INFO [RS:1;jenkins-hbase4:38051] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 16:56:13,253 INFO [RS:1;jenkins-hbase4:38051] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 16:56:13,255 INFO [RS:1;jenkins-hbase4:38051] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38051 2023-02-13 16:56:13,258 DEBUG [RS:2;jenkins-hbase4:34717] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/oldWALs 2023-02-13 16:56:13,258 INFO [RS:2;jenkins-hbase4:34717] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34717%2C1676307263076:(num 1676307264541) 2023-02-13 16:56:13,258 DEBUG [RS:2;jenkins-hbase4:34717] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:13,258 INFO [RS:2;jenkins-hbase4:34717] regionserver.LeaseManager(133): Closed leases 2023-02-13 16:56:13,258 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:13,258 INFO [RS:2;jenkins-hbase4:34717] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 16:56:13,258 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38051,1676307263041 2023-02-13 16:56:13,258 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 16:56:13,259 INFO [RS:2;jenkins-hbase4:34717] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34717 2023-02-13 16:56:13,261 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34717,1676307263076 2023-02-13 16:56:13,261 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2e6357f7 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,264 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 16:56:13,264 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38051,1676307263041] 2023-02-13 16:56:13,264 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38051,1676307263041; numProcessing=2 2023-02-13 16:56:13,266 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38051,1676307263041 already deleted, retry=false 2023-02-13 16:56:13,266 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38051,1676307263041 expired; onlineServers=1 2023-02-13 16:56:13,266 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34717,1676307263076] 2023-02-13 16:56:13,266 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34717,1676307263076; numProcessing=3 2023-02-13 16:56:13,268 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34717,1676307263076 already deleted, retry=false 2023-02-13 16:56:13,268 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34717,1676307263076 expired; onlineServers=0 2023-02-13 16:56:13,268 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,45273,1676307261815' ***** 2023-02-13 16:56:13,268 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 16:56:13,269 DEBUG [M:0;jenkins-hbase4:45273] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7dfcd593, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 16:56:13,269 INFO [M:0;jenkins-hbase4:45273] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,45273,1676307261815 2023-02-13 16:56:13,269 INFO [M:0;jenkins-hbase4:45273] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,45273,1676307261815; all regions closed. 2023-02-13 16:56:13,269 DEBUG [M:0;jenkins-hbase4:45273] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:13,269 DEBUG [M:0;jenkins-hbase4:45273] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 16:56:13,269 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 16:56:13,269 DEBUG [M:0;jenkins-hbase4:45273] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 16:56:13,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676307264162] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676307264162,5,FailOnTimeoutGroup] 2023-02-13 16:56:13,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676307264161] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676307264161,5,FailOnTimeoutGroup] 2023-02-13 16:56:13,270 INFO [M:0;jenkins-hbase4:45273] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 16:56:13,270 INFO [M:0;jenkins-hbase4:45273] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 16:56:13,270 INFO [M:0;jenkins-hbase4:45273] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-13 16:56:13,270 DEBUG [M:0;jenkins-hbase4:45273] master.HMaster(1502): Stopping service threads 2023-02-13 16:56:13,270 INFO [M:0;jenkins-hbase4:45273] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 16:56:13,270 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 16:56:13,270 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:56:13,270 ERROR [M:0;jenkins-hbase4:45273] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-13 16:56:13,270 INFO [M:0;jenkins-hbase4:45273] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:13,270 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 16:56:13,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 16:56:13,271 DEBUG [M:0;jenkins-hbase4:45273] zookeeper.ZKUtil(398): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 16:56:13,271 WARN [M:0;jenkins-hbase4:45273] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 16:56:13,271 INFO [M:0;jenkins-hbase4:45273] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:13,271 INFO [M:0;jenkins-hbase4:45273] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 16:56:13,271 DEBUG [M:0;jenkins-hbase4:45273] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 16:56:13,271 INFO [M:0;jenkins-hbase4:45273] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:56:13,271 DEBUG [M:0;jenkins-hbase4:45273] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:56:13,271 DEBUG [M:0;jenkins-hbase4:45273] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 16:56:13,271 DEBUG [M:0;jenkins-hbase4:45273] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:56:13,271 INFO [M:0;jenkins-hbase4:45273] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.17 KB heapSize=719.09 KB 2023-02-13 16:56:13,286 INFO [M:0;jenkins-hbase4:45273] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.17 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/4c04911dce3f433ebd4768f9762e8633 2023-02-13 16:56:13,290 INFO [M:0;jenkins-hbase4:45273] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4c04911dce3f433ebd4768f9762e8633 2023-02-13 16:56:13,290 DEBUG [M:0;jenkins-hbase4:45273] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/4c04911dce3f433ebd4768f9762e8633 as hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4c04911dce3f433ebd4768f9762e8633 2023-02-13 16:56:13,293 INFO [M:0;jenkins-hbase4:45273] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4c04911dce3f433ebd4768f9762e8633 2023-02-13 16:56:13,294 INFO [M:0;jenkins-hbase4:45273] regionserver.HStore(1080): Added hdfs://localhost:33763/user/jenkins/test-data/afd5bf36-37a1-0b1b-afb4-dd980ceff07c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4c04911dce3f433ebd4768f9762e8633, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-13 16:56:13,294 INFO [M:0;jenkins-hbase4:45273] regionserver.HRegion(2947): Finished flush of dataSize ~608.17 KB/622769, heapSize ~719.08 KB/736336, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=1282, compaction requested=false 2023-02-13 16:56:13,295 INFO [M:0;jenkins-hbase4:45273] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:56:13,295 DEBUG [M:0;jenkins-hbase4:45273] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676307373271Disabling compacts and flushes for region at 1676307373271Disabling writes for close at 1676307373271Obtaining lock to block concurrent updates at 1676307373271Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676307373271Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622769, getHeapSize=736336, getOffHeapSize=0, getCellsCount=1499 at 1676307373272 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676307373272Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676307373272Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676307373276 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676307373276Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676307373291 (+15 ms)Finished flush of dataSize ~608.17 KB/622769, heapSize ~719.08 KB/736336, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=1282, compaction requested=false at 1676307373294 (+3 ms)Writing region close event to WAL at 1676307373295 (+1 ms)Closed at 1676307373295 2023-02-13 16:56:13,302 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 16:56:13,302 INFO [M:0;jenkins-hbase4:45273] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 16:56:13,303 INFO [M:0;jenkins-hbase4:45273] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45273 2023-02-13 16:56:13,304 DEBUG [M:0;jenkins-hbase4:45273] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,45273,1676307261815 already deleted, retry=false 2023-02-13 16:56:13,361 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:13,361 INFO [RS:1;jenkins-hbase4:38051] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38051,1676307263041; zookeeper connection closed. 2023-02-13 16:56:13,361 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5e7634b6 rejected from java.util.concurrent.ThreadPoolExecutor@5a49bba0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,361 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:38051-0x1004e9acff70002, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:13,361 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@41225ae4 rejected from java.util.concurrent.ThreadPoolExecutor@5a49bba0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,361 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@671a383a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@671a383a 2023-02-13 16:56:13,461 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:13,461 INFO [M:0;jenkins-hbase4:45273] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,45273,1676307261815; zookeeper connection closed. 2023-02-13 16:56:13,461 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5b1276f rejected from java.util.concurrent.ThreadPoolExecutor@1d7a5f33[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,461 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): master:45273-0x1004e9acff70000, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:13,461 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@45b4b60e rejected from java.util.concurrent.ThreadPoolExecutor@1d7a5f33[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,561 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:13,561 INFO [RS:2;jenkins-hbase4:34717] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,34717,1676307263076; zookeeper connection closed. 2023-02-13 16:56:13,561 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6a933d91 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,561 DEBUG [Listener at localhost/38547-EventThread] zookeeper.ZKWatcher(600): regionserver:34717-0x1004e9acff70003, quorum=127.0.0.1:57428, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:13,561 ERROR [Listener at localhost/38547-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@125c2d39 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:13,561 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@57f095d5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@57f095d5 2023-02-13 16:56:13,562 INFO [Listener at localhost/38547] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 16:56:13,562 WARN [Listener at localhost/38547] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 16:56:13,565 INFO [Listener at localhost/38547] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 16:56:13,673 WARN [BP-2104803542-172.31.14.131-1676307258605 heartbeating to localhost/127.0.0.1:33763] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2104803542-172.31.14.131-1676307258605 (Datanode Uuid a3585e58-faa4-4a64-84b5-04c6ddf75562) service to localhost/127.0.0.1:33763 2023-02-13 16:56:13,675 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data5/current/BP-2104803542-172.31.14.131-1676307258605] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 16:56:13,675 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data6/current/BP-2104803542-172.31.14.131-1676307258605] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 16:56:13,676 WARN [Listener at localhost/38547] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 16:56:13,678 INFO [Listener at localhost/38547] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 16:56:13,783 WARN [BP-2104803542-172.31.14.131-1676307258605 heartbeating to localhost/127.0.0.1:33763] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 16:56:13,783 WARN [BP-2104803542-172.31.14.131-1676307258605 heartbeating to localhost/127.0.0.1:33763] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2104803542-172.31.14.131-1676307258605 (Datanode Uuid 14571ecd-b0a5-4d58-a36d-d2e6e3cf032c) service to localhost/127.0.0.1:33763 2023-02-13 16:56:13,784 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data3/current/BP-2104803542-172.31.14.131-1676307258605] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 16:56:13,784 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data4/current/BP-2104803542-172.31.14.131-1676307258605] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 16:56:13,784 WARN [Listener at localhost/38547] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 16:56:13,786 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:56:13,786 INFO [Listener at localhost/38547] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 16:56:13,888 WARN [BP-2104803542-172.31.14.131-1676307258605 heartbeating to localhost/127.0.0.1:33763] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2104803542-172.31.14.131-1676307258605 (Datanode Uuid 1fadd7f6-283b-4c52-81b3-6a5231ac6c2e) service to localhost/127.0.0.1:33763 2023-02-13 16:56:13,889 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data1/current/BP-2104803542-172.31.14.131-1676307258605] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 16:56:13,889 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/29a42920-968a-61ad-d8ad-feccd5d020e4/cluster_3bc55ca0-f001-8497-ffb8-8a166e5ed134/dfs/data/data2/current/BP-2104803542-172.31.14.131-1676307258605] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 16:56:13,909 INFO [Listener at localhost/38547] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 16:56:14,098 INFO [Listener at localhost/38547] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 16:56:14,344 INFO [Listener at localhost/38547] hbase.HBaseTestingUtility(1293): Minicluster is down